{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pomegranate Tutorial 1: Distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "author: Jacob Schreiber <br>\n",
    "contact: jmschreiber91@gmail.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will cover the various distributions that are implemented in pomegranate. These distributions can be used by themselves, or more typically as a part of a larger compositional model like a hidden Markov model. \n",
    "\n",
    "The distributions that are currently available:\n",
    "\n",
    "- Univariate Distributions\n",
    "  1. UniformDistribution\n",
    "  2. BernoulliDistribution\n",
    "  2. NormalDistribution\n",
    "  3. LogNormalDistribution\n",
    "  4. ExponentialDistribution\n",
    "  5. PoissonDistribution\n",
    "  6. BetaDistribution\n",
    "  7. GammaDistribution\n",
    "  8. DiscreteDistribution\n",
    "  \n",
    "- Univariate Kernel Densities\n",
    "  1. GaussianKernelDensity\n",
    "  2. UniformKernelDensity\n",
    "  3. TriangleKernelDensity\n",
    "  \n",
    "- Multivariate Distributions\n",
    "  1. IndependentComponentsDistribution\n",
    "  2. Dirichlet Distribution\n",
    "  3. MultivariateGaussianDistribution\n",
    "  4. ConditionalProbabilityTable\n",
    "  5. JointProbabilityTable\n",
    "\n",
    "The following methods are supported by all distributions:\n",
    "\n",
    "```\n",
    "1. d.probability(X): Return the probability of a sample under the distribution or a vector of samples\n",
    "2. d.log_probability(X): Return the log probability of a sample under the distribution or a vector of samples\n",
    "3. d.fit(X, weights=None, inertia): Use MLE estimates to update the parameters of the distribution. Optional weight vector is accepted, or inertia that pushes parameters from where they currently are to the update that portion of the way.\n",
    "4. d.summarize(X, weights=None): Extract and update the stored sufficient statistics on a data set with optional weights, but don't update the parameters yet\n",
    "5. d.from_summaries(X, inertia=0.0): Use the stored sufficient statistics to do a parameter update\n",
    "6. d.sample(n=1): Generate a sample, or a vector of samples, from the distribution\n",
    "\n",
    "7. d.to_json(): Serialize the distribution to a string json\n",
    "8. Distribution.from_json(s): Load up a stored distriibution, either from a file if passed a filename that ends in '.json', or from the string passed in itself.\n",
    "\n",
    "9. d.copy(): Make a deep copy of the distribution\n",
    "10. d.freeze(): Freeze the distribution, making 'fit' and 'from_summaries' not change model parameters\n",
    "11. d.thaw(): Unfreeze the distribution so that it can be updated agin.\n",
    "```\n",
    "\n",
    "Univariate distributions also have a `plot( n=1000, **kwargs )` command, where you can plot a histogram of `n` randomly generated samples from the distribution, which will pass matplotlib keyword arguments to the `plt.hist` function. \n",
    "\n",
    "Lets look at a few examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Normal Distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "from pomegranate import *\n",
    "%pylab inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets start off by looking at a normal distribution. It is easy to specify, and take a look at."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFVlJREFUeJzt3X+s3fV93/Hnq/xI1jSN7WB+1DY1\nUb2syaQm7AromKosbg14VcwfYXNUjTuK5FUjU6JNWmGdZg2SKdmkZkXaqKzgzVRZgNIyrIiUuE5Q\nVSQIhgABHGZDE7Ds2G7tmDDUFNL3/jifG47Nub7n2veec6+/z4d0dL7f9/dzzv18zzn3+zrfH+f7\nTVUhSeqenxp3ByRJ42EASFJHGQCS1FEGgCR1lAEgSR1lAEhSRxkAktRRBoAkdZQBIEkddfa4O3Ay\n5513Xq1evXrc3ZCkReWJJ574y6paPlO7BR0Aq1evZteuXePuhiQtKkm+N0y7GTcBJXl/kqf6bq8m\n+XSSZUl2JNnT7pe29klye5K9SZ5Jcmnfc0229nuSTJ767EmSTteMAVBVL1TVh6rqQ8A/AF4H7gdu\nBnZW1RpgZxsHuAZY026bgDsAkiwDNgOXA5cBm6dCQ5I0erPdCbwWeLGqvgdsALa1+jbg2ja8Abir\neh4FliS5CLgK2FFVR6rqKLADuPq050CSdEpmGwAbgS+34Quq6gBAuz+/1VcAr/Q9Zl+rTVc/TpJN\nSXYl2XX48OFZdk+SNKyhAyDJucDHgD+cqemAWp2kfnyhaktVTVTVxPLlM+7EliSdotmsAVwDPFlV\nB9v4wbZph3Z/qNX3Aav6HrcS2H+SuiRpDGYTAJ/grc0/ANuBqSN5JoEH+urXt6OBrgCOtU1EDwHr\nkixtO3/XtZokaQyG+h1Akp8Gfg34l33lzwH3JrkReBm4rtUfBNYDe+kdMXQDQFUdSXIb8Hhrd2tV\nHTntOZAknZIs5GsCT0xMlD8Ek6TZSfJEVU3M1G5B/xJYWowufOQRDr7xxtvqF5xzDt+/8sox9Ega\nzACQZjDbBfqgtierS+NiAEgzmMsFeh5++G011ww0Lp4OWhoz1ww0Lq4BSKdh0Dd6abFwDUCSOsoA\nkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpozwXkLQAeJZQjYNr\nANIC5VlCNd8MAEnqKANAkjrKfQBSM92lH6Uz1VBrAEmWJLkvyXeS7E7yy0mWJdmRZE+7X9raJsnt\nSfYmeSbJpX3PM9na70kyOV8zJZ0KF/7qmmE3Af0e8CdV9feAXwJ2AzcDO6tqDbCzjQNcA6xpt03A\nHQBJlgGbgcuBy4DNU6EhSRq9GQMgyc8CvwLcCVBVf1NVPwA2ANtas23AtW14A3BX9TwKLElyEXAV\nsKOqjlTVUWAHcPWczo0kaWjDrAG8DzgM/M8k30ryxSTvAi6oqgMA7f781n4F8Erf4/e12nR1SdIY\nDBMAZwOXAndU1YeB/8dbm3sGyYBanaR+/IOTTUl2Jdl1+PDhIbonSToVwwTAPmBfVT3Wxu+jFwgH\n26Yd2v2hvvar+h6/Eth/kvpxqmpLVU1U1cTy5ctnMy+SpFmYMQCq6vvAK0ne30prgeeB7cDUkTyT\nwANteDtwfTsa6ArgWNtE9BCwLsnStvN3XatJksZg2N8B/GvgS0nOBV4CbqAXHvcmuRF4GbiutX0Q\nWA/sBV5vbamqI0luAx5v7W6tqiNzMheSpFkbKgCq6ilgYsCktQPaFnDTNM+zFdg6mw5KkuaHvwSW\nFjDPEqr55LmApEXGXyxrrhgAktRRBoAkdZQBIEkd5U5gdYqnfJbe4hqAOsWFv/QWA0CSOsoAkKSO\nMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSO\nGup6AEm+C/wQ+DHwZlVNJFkG3AOsBr4L/NOqOpokwO8B64HXgX9RVU+255kE/kN72s9U1ba5mxWp\nO7xYvObCbNYA/nFVfaiqJtr4zcDOqloD7GzjANcAa9ptE3AHQAuMzcDlwGXA5iRLT38WJIHXOtDs\nnc4moA3A1Df4bcC1ffW7qudRYEmSi4CrgB1VdaSqjgI7gKtP4+9Lkk7DsAFQwNeSPJFkU6tdUFUH\nANr9+a2+Anil77H7Wm26uiRpDIa9JvCVVbU/yfnAjiTfOUnbDKjVSerHP7gXMJsALr744iG7J0ma\nraECoKr2t/tDSe6ntw3/YJKLqupA28RzqDXfB6zqe/hKYH+rf+SE+sMD/tYWYAvAxMTE2wJCGoYX\nf5dmNuMmoCTvSvLuqWFgHfAssB2YbM0mgQfa8Hbg+vRcARxrm4geAtYlWdp2/q5rNWnOufCXZjbM\nGsAFwP29ozs5G/jfVfUnSR4H7k1yI/AycF1r/yC9Q0D30jsM9AaAqjqS5Dbg8dbu1qo6MmdzIkma\nlRkDoKpeAn5pQP2vgLUD6gXcNM1zbQW2zr6bkqS55i+BJamjDABJ6igDQJI6ygCQpI4yACSpowwA\nSeooA0CSOsoAkKSOMgAkqaMMAEnqKANAkjrKAJCkjhr2gjCSFgEvFq/ZcA1AOsN5bQRNxzUALWpe\n+Us6da4BaFFz4S+dOgNAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4aOgCSnJXkW0m+0sYvSfJY\nkj1J7klybqu/o43vbdNX9z3HLa3+QpKr5npmJEnDm80awKeA3X3jnwe+UFVrgKPAja1+I3C0qn4B\n+EJrR5IPABuBDwJXA/8jyVmn131J0qkaKgCSrAT+CfDFNh7go8B9rck24No2vKGN06avbe03AHdX\n1Y+q6i+AvcBlczETkqTZG3YN4L8B/w742zb+XuAHVfVmG98HrGjDK4BXANr0Y639T+oDHvMTSTYl\n2ZVk1+HDh2cxK5Kk2ZgxAJL8OnCoqp7oLw9oWjNMO9lj3ipUbamqiaqaWL58+UzdkySdomFOBncl\n8LEk64F3Aj9Lb41gSZKz27f8lcD+1n4fsArYl+Rs4D3Akb76lP7HSJJGbMY1gKq6papWVtVqejtx\nv15VvwF8A/h4azYJPNCGt7dx2vSvV1W1+sZ2lNAlwBrgm3M2J5KkWTmd00H/NnB3ks8A3wLubPU7\ngT9IspfeN/+NAFX1XJJ7geeBN4GbqurHp/H3JUmnYVYBUFUPAw+34ZcYcBRPVf01cN00j/8s8NnZ\ndlKSNPf8JbAkdZQBIEkdZQBIUkd5TWCpA/Lww2+rXXDOOXz/yitH3xktGK4BSB3l9ZRlAEhSRxkA\nktRR7gPQonDhI4+4yUKaY64BaFFw4S/NPQNAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSp\nowwASeooA0CSOsoAkKSOMgAkqaMMAEnqqBkDIMk7k3wzydNJnkvyn1r9kiSPJdmT5J4k57b6O9r4\n3jZ9dd9z3dLqLyS5ar5mSpI0s2FOB/0j4KNV9VqSc4A/T/JV4N8AX6iqu5P8PnAjcEe7P1pVv5Bk\nI/B54J8l+QCwEfgg8HPAnyb5u1X143mYL0lDGHSpSPBykV0x4xpA9bzWRs9ptwI+CtzX6tuAa9vw\nhjZOm742SVr97qr6UVX9BbAXuGxO5kLSnPL0290w1D6AJGcleQo4BOwAXgR+UFVvtib7gBVteAXw\nCkCbfgx4b399wGMkSSM21BXB2maaDyVZAtwP/OKgZu0+00ybrn6cJJuATQAXX3zxMN3TGcQrf0mj\nM6ujgKrqB8DDwBXAkiRTAbIS2N+G9wGrANr09wBH+usDHtP/N7ZU1URVTSxfvnw23dMZwIW/NDrD\nHAW0vH3zJ8nfAX4V2A18A/h4azYJPNCGt7dx2vSvV1W1+sZ2lNAlwBrgm3M1I5Kk2RlmE9BFwLYk\nZ9ELjHur6itJngfuTvIZ4FvAna39ncAfJNlL75v/RoCqei7JvcDzwJvATR4BJEnjM2MAVNUzwIcH\n1F9iwFE8VfXXwHXTPNdngc/OvpuSpLnmL4ElqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6\nygCQpI4yACSpowwASeqooU4HLal7Bl0tzCuFnVlcA5A0NE/XfWYxACSpowwASeooA0CSOsoAkKSO\nMgAkqaM8DFRjceEjj3hEiTRmrgFoLFz4S+NnAEhSRxkAktRRMwZAklVJvpFkd5Lnknyq1Zcl2ZFk\nT7tf2upJcnuSvUmeSXJp33NNtvZ7kkzO32xJkmYyzBrAm8C/rapfBK4AbkryAeBmYGdVrQF2tnGA\na4A17bYJuAN6gQFsBi4HLgM2T4WGJGn0ZgyAqjpQVU+24R8Cu4EVwAZgW2u2Dbi2DW8A7qqeR4El\nSS4CrgJ2VNWRqjoK7ACuntO5kSQNbVb7AJKsBj4MPAZcUFUHoBcSwPmt2Qrglb6H7Wu16eqSpDEY\nOgCS/AzwR8Cnq+rVkzUdUKuT1E/8O5uS7Eqy6/Dhw8N2T5I0S0MFQJJz6C38v1RVf9zKB9umHdr9\noVbfB6zqe/hKYP9J6sepqi1VNVFVE8uXL5/NvEiSZmGYo4AC3Ansrqrf7Zu0HZg6kmcSeKCvfn07\nGugK4FjbRPQQsC7J0rbzd12rSZLGYJhTQVwJ/HPg20mearV/D3wOuDfJjcDLwHVt2oPAemAv8Dpw\nA0BVHUlyG/B4a3drVR2Zk7mQJM3ajAFQVX/O4O33AGsHtC/gpmmeayuwdTYdlCTND08GJ2lWvFbw\nmcNTQUg6bZ7cb3FyDUDzytM+SwuXawCaVy78pYXLAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSp\nowwASeooA0CSOsoAkKSO8lQQkuaEJ4lbfFwDkDRvPBXIwmYASFJHGQCS1FEGgCR1lDuBNSc877+0\n+LgGoDnhwl9afAwASeooA0CSOmrGAEiyNcmhJM/21ZYl2ZFkT7tf2upJcnuSvUmeSXJp32MmW/s9\nSSbnZ3YkScMaZg3gfwFXn1C7GdhZVWuAnW0c4BpgTbttAu6AXmAAm4HLgcuAzVOhIUkajxkDoKr+\nDDhyQnkDsK0NbwOu7avfVT2PAkuSXARcBeyoqiNVdRTYwdtDRZI0Qqd6GOgFVXUAoKoOJDm/1VcA\nr/S129dq09XfJskmemsPXHzxxafYPUkLhecIWrjmeidwBtTqJPW3F6u2VNVEVU0sX758TjsnaWHw\nsOGF4VQD4GDbtEO7P9Tq+4BVfe1WAvtPUpckjcmpbgLaDkwCn2v3D/TVP5nkbno7fI+1TUQPAf+5\nb8fvOuCWU++2xsVf/EpnjhkDIMmXgY8A5yXZR+9ons8B9ya5EXgZuK41fxBYD+wFXgduAKiqI0lu\nAx5v7W6tqhN3LGsRcOEvnTlmDICq+sQ0k9YOaFvATdM8z1Zg66x6J0maN/4SWJI6ygCQpI4yACSp\nowwASeooLwijgTzcU/PNXwiPn2sAGsiFv8bBz91oGQCS1FEGgCR1lAEgSR1lAEhSRxkAktRRHgYq\naUHx8NDRMQA6zuP9tRj4GZ0fbgLqOP+xpO4yACSpo9wEJGlRcN/A3HMNQNKi5SbM0+MaQEe4s1fS\niVwD6AgX/pJO5BrAGcRv+eoi9w2cOgPgDOLCX+o5+MYbBsMQRr4JKMnVSV5IsjfJzaP++5K6yy9J\nxxvpGkCSs4D/DvwasA94PMn2qnp+lP1Y7NzUI5061wzeMupNQJcBe6vqJYAkdwMbAANgABf00mh0\ndZPRqANgBfBK3/g+4PIR92HeTbfg/ingbwe0n64uabymCwaY/v92MYXGqAMgA2p1XINkE7Cpjb6W\n5IV579XpOw/4y5kaTbeQH8PCf6j+LiCLrb+w+Ppsf2dpuv/bgwxc0I26vz8/TKNRB8A+YFXf+Epg\nf3+DqtoCbBllp05Xkl1VNTHufgzL/s6/xdZn+zu/Fmp/R30U0OPAmiSXJDkX2AhsH3EfJEmMeA2g\nqt5M8kngIeAsYGtVPTfKPkiSekb+Q7CqehB4cNR/d54tqk1W2N9RWGx9tr/za0H2N1U1cytJ0hnH\nk8FJUkcZAKcgyX9N8p0kzyS5P8mSadp9N8m3kzyVZNcY+nnS024keUeSe9r0x5KsHnUf+/qyKsk3\nkuxO8lySTw1o85Ekx9rr+VSS/ziOvvb156Tvb3pub6/vM0kuHUc/W1/e3/e6PZXk1SSfPqHN2F/f\nJFuTHErybF9tWZIdSfa0+6XTPHaytdmTZHKM/V0UywcAqsrbLG/AOuDsNvx54PPTtPsucN6Y+ngW\n8CLwPuBc4GngAye0+VfA77fhjcA9Y3xNLwIubcPvBv7vgP5+BPjKuN//Yd9fYD3wVXqHhV8BPDbu\nPvd9Nr4P/PxCe32BXwEuBZ7tq/0X4OY2fPOg/zdgGfBSu1/ahpeOqb8LfvkwdXMN4BRU1deq6s02\n+ii93zMsND857UZV/Q0wddqNfhuAbW34PmBtkkE/1pt3VXWgqp5swz8EdtP75fhitgG4q3oeBZYk\nuWjcnQLWAi9W1ffG3ZETVdWfAUdOKPd/TrcB1w546FXAjqo6UlVHgR3A1fPW0WZQfxfJ8gFwE9Bc\n+E163/IGKeBrSZ5ov3AepUGn3ThxgfqTNu0Dewx470h6dxJtU9SHgccGTP7lJE8n+WqSD460Y283\n0/s7zHswDhuBL08zbSG9vlMuqKoD0PuiAJw/oM1Cfa0X6vIB8HoA00ryp8CFAyb9TlU90Nr8DvAm\n8KVpnubKqtqf5HxgR5LvtG8MozDjaTeGbDNSSX4G+CPg01X16gmTn6S32eK1JOuB/wOsGXUf+8z0\n/i7E1/dc4GPALQMmL7TXdzYW4mu9kJcPgGsA06qqX62qvz/gNrXwnwR+HfiNahv0BjzH/nZ/CLif\n3maZUZnxtBv9bZKcDbyHt69+j0ySc+gt/L9UVX984vSqerWqXmvDDwLnJDlvxN3s789M7+8w78Go\nXQM8WVUHT5yw0F7fPgenNp21+0MD2iyo13oRLB8AA+CUJLka+G3gY1X1+jRt3pXk3VPD9HYMPTuo\n7TwZ5rQb24GpoyU+Dnx9ug/rfGv7Hu4EdlfV707T5sKpfRRJLqP3+f2r0fXyuL4M8/5uB65vRwNd\nARyb2pQxRp9gms0/C+n1PUH/53QSeGBAm4eAdUmWtqOE1rXayC2S5UPPOPdAL9YbsJfe9san2m3q\nSJqfAx5sw++jd+TN08Bz9DYdjbqf6+kdTfPi1N8HbqX3wQR4J/CHbX6+CbxvjK/pP6K3yv5M3+u6\nHvgt4Ldam0+21/JpejvX/uEY+zvw/T2hv6F3AaQXgW8DE2P+3P40vQX6e/pqC+r1pRdOB4A36H2r\nv5HefqmdwJ52v6y1nQC+2PfY32yf5b3ADWPs76JYPlSVvwSWpK5yE5AkdZQBIEkdZQBIUkcZAJLU\nUQaAJHWUASBJHWUASFJHGQCS1FH/HyJPWx0+o78/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fae34260150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = NormalDistribution( 5, 2 )\n",
    "d.plot( n=100000, edgecolor='c', color='c', bins=50 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then calculate the log probability of some points under this distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-3.612085713764219\n",
      "-1.612085713764219\n",
      "-3.612085713764219\n"
     ]
    }
   ],
   "source": [
    "print d.log_probability( 1 )\n",
    "print d.log_probability( 5 )\n",
    "print d.log_probability( 9 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also fit this distribution to some data we have."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = np.random.randn(10000) * 0.4 + 9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[9.004974667786882, 0.39447571955036087]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFYZJREFUeJzt3X2sXPWd3/H3J8QhD22DAeNQ26zZ\nxsomqRrCWuBdS1Ea7xpDK0zbIJE+4FBX7qrsKqkqtaR/FC1s1KSqmiztLis3uDXRNoSlm+JGbFjX\nibstEgSTEB6D7JANuDbGix3SCG0Kybd/zM8wXOb6ztj3ztx7z/sljeac7/nN3O+5mjufex7mTKoK\nSVL3vGnSDUiSJsMAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI66s2TbuBkzj33\n3Fq9evWk25CkBeWhhx76s6paNtO4eR0Aq1evZt++fZNuQ5IWlCQ/GGacu4AkqaMMAEnqqBkDIMl7\nkjzcd/tRkk8mOTvJ7iT72/3SNj5JbklyIMkjSS7ue64tbfz+JFvmcsUkSSc3YwBU1VNVdVFVXQT8\nIvAS8BXgBmBPVa0B9rR5gMuBNe22DbgVIMnZwI3ApcAlwI0nQkOSNH6j7gLaAHyvqn4AbAZ2tvpO\n4Ko2vRm4vXruB85Kcj5wGbC7qo5V1XFgN7DptNdAknRKRg2Aa4AvtenlVXUYoN2f1+orgGf7HnOw\n1aarS5ImYOgASPIW4ErgD2YaOqBWJ6lP/TnbkuxLsu/o0aPDtidJGtEoWwCXA9+qqiNt/kjbtUO7\nf77VDwKr+h63Ejh0kvrrVNX2qlpbVWuXLZvxcwySpFM0SgB8jNd2/wDsAk6cybMFuLuvfm07G2gd\n8GLbRXQvsDHJ0nbwd2OrSZImYKhPAid5O/CrwD/uK38GuDPJVuAZ4OpWvwe4AjhA74yh6wCq6liS\nm4EH27ibqurYaa+BNMfedd99HHn55TfU3wT8bMD46erLlyzhufXrZ7k76dQNFQBV9RJwzpTaC/TO\nCpo6toDrp3meHcCO0duUJmfQmz8MfpM/WX2655EmxU8CS1JHGQCS1FEGgCR1lAEgSR1lAEhSR83r\nL4SRxmm60z2lxcotAKnxzV9dYwBIUkcZAJLUUQaAJHWUASBJHeVZQNIYZe/eN9S8SJwmxS0AacI8\n+0iTYgBIUkcZAJLUUQaAJHWUASBJHWUASFJHeRqoOsULvkmvcQtAneKbv/QaA0CSOmqoAEhyVpK7\nknw3yZNJfinJ2Ul2J9nf7pe2sUlyS5IDSR5JcnHf82xp4/cn2TJXKyVJmtmwWwC/DXytqn4B+ADw\nJHADsKeq1gB72jzA5cCadtsG3AqQ5GzgRuBS4BLgxhOhIUkavxkDIMlfAj4E3AZQVf+vqn4IbAZ2\ntmE7gava9Gbg9uq5HzgryfnAZcDuqjpWVceB3cCmWV0bSdLQhtkC+HngKPCfknw7yReSvANYXlWH\nAdr9eW38CuDZvscfbLXp6pKkCRjmNNA3AxcDv1FVDyT5bV7b3TNIBtTqJPXXPzjZRm/XERdccMEQ\n7UkLn1cJ1SQMswVwEDhYVQ+0+bvoBcKRtmuHdv983/hVfY9fCRw6Sf11qmp7Va2tqrXLli0bZV2k\nRcVTVjXXZgyAqnoOeDbJe1ppA/AEsAs4cSbPFuDuNr0LuLadDbQOeLHtIroX2JhkaTv4u7HVJEkT\nMOwngX8D+P0kbwGeBq6jFx53JtkKPANc3cbeA1wBHABeamOpqmNJbgYebONuqqpjs7IWkqSRDRUA\nVfUwsHbAog0DxhZw/TTPswPYMUqDkqS54SeBJamjDABJ6igDQJI6ygCQpI4yACSpo/xCGC1KfvGL\nNDO3ALQo+eYvzcwAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSp\nowwASeooA0CSOsqrgUrzWPbufUNt+ZIlPLd+/fib0aJjAGhB6+Jln7u2vpo7Q+0CSvKnSR5N8nCS\nfa12dpLdSfa3+6WtniS3JDmQ5JEkF/c9z5Y2fn+SLXOzSuoS3wylUzfKMYC/XlUXVdXaNn8DsKeq\n1gB72jzA5cCadtsG3Aq9wABuBC4FLgFuPBEakqTxO52DwJuBnW16J3BVX/326rkfOCvJ+cBlwO6q\nOlZVx4HdwKbT+PmSpNMwbAAU8MdJHkqyrdWWV9VhgHZ/XquvAJ7te+zBVpuuLkmagGEPAq+vqkNJ\nzgN2J/nuScZmQK1OUn/9g3sBsw3gggsuGLI9SdKohtoCqKpD7f554Cv09uEfabt2aPfPt+EHgVV9\nD18JHDpJferP2l5Va6tq7bJly0ZbG0nS0GYMgCTvSPIXT0wDG4HHgF3AiTN5tgB3t+ldwLXtbKB1\nwIttF9G9wMYkS9vB342tJkmagGF2AS0HvpLkxPj/UlVfS/IgcGeSrcAzwNVt/D3AFcAB4CXgOoCq\nOpbkZuDBNu6mqjo2a2siSRrJjAFQVU8DHxhQfwHYMKBewPXTPNcOYMfobUqSZpvXApKkjjIAJKmj\nDABJ6igDQJI6ygCQpI4yACSpowwASeooA0CSOsoAkKSOMgAkqaMMAEnqKL8UXlqAsnfvG2rLlyzh\nufXrx9+MFiy3AKRF4sjLL0+6BS0wbgFoQXjXfff5BifNMrcAtCD45i/NPgNAkjrKAJCkjjIAJKmj\nDABJ6igDQJI6ygCQpI4aOgCSnJHk20m+2uYvTPJAkv1JvpzkLa1+Zps/0Jav7nuOT7X6U0kum+2V\nkSQNb5QtgE8AT/bNfxb4XFWtAY4DW1t9K3C8qt4NfK6NI8n7gGuA9wObgN9NcsbptS9JOlVDBUCS\nlcDfAL7Q5gN8BLirDdkJXNWmN7d52vINbfxm4I6q+klVfR84AFwyGyshSRrdsFsAnwf+OfCzNn8O\n8MOqeqXNHwRWtOkVwLMAbfmLbfyr9QGPkSSN2YwBkORvAs9X1UP95QFDa4ZlJ3tM/8/blmRfkn1H\njx6dqT1J0ikaZgtgPXBlkj8F7qC36+fzwFlJTlxMbiVwqE0fBFYBtOXvBI711wc85lVVtb2q1lbV\n2mXLlo28QpKk4cwYAFX1qapaWVWr6R3E/XpV/T3gG8BH27AtwN1telebpy3/elVVq1/TzhK6EFgD\nfHPW1kSSNJLTuRz0vwDuSPJbwLeB21r9NuCLSQ7Q+8//GoCqejzJncATwCvA9VX109P4+ZKk0zBS\nAFTVXmBvm36aAWfxVNWfA1dP8/hPA58etUlJ0uzzk8CS1FEGgCR1lAEgSR3ldwJLi0j27n1DbfmS\nJTy3fv34m9G85xaAtMj5fcqajgEgSR1lAEhSRxkAktRRBoAkdZQBIEkd5Wmgmlfedd99nrUijYlb\nAJpXfPOXxscAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI6aMQCSvDXJ\nN5N8J8njSX6z1S9M8kCS/Um+nOQtrX5mmz/Qlq/ue65PtfpTSS6bq5WSJM1smC2AnwAfqaoPABcB\nm5KsAz4LfK6q1gDHga1t/FbgeFW9G/hcG0eS9wHXAO8HNgG/m+SM2VwZSdLwZgyA6vlxm13SbgV8\nBLir1XcCV7XpzW2etnxDkrT6HVX1k6r6PnAAuGRW1kKSNLKhrgba/lN/CHg38DvA94AfVtUrbchB\nYEWbXgE8C1BVryR5ETin1e/ve9r+x0iaQ35ZvAYZ6iBwVf20qi4CVtL7r/29g4a1+0yzbLr66yTZ\nlmRfkn1Hjx4dpj1Jp8Arr2qks4Cq6ofAXmAdcFaSE1sQK4FDbfogsAqgLX8ncKy/PuAx/T9je1Wt\nraq1y5YtG6U9SdIIhjkLaFmSs9r024BfAZ4EvgF8tA3bAtzdpne1edryr1dVtfo17SyhC4E1wDdn\na0UkSaMZ5hjA+cDOdhzgTcCdVfXVJE8AdyT5LeDbwG1t/G3AF5McoPef/zUAVfV4kjuBJ4BXgOur\n6qezuzqSpGHNGABV9QjwwQH1pxlwFk9V/Tlw9TTP9Wng06O3KUmabX4nsCbC7/6VJs9LQWgifPOX\nJs8AkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6ygCQpI4yACSpowwASeooLwYn\nddigr4oEvy6yK9wCkPQGXqyvGwwASeooA0CSOspjAJpTfvGLNH+5BaA55Zu/NH8ZAJLUUQaAJHXU\njAGQZFWSbyR5MsnjST7R6mcn2Z1kf7tf2upJckuSA0keSXJx33NtaeP3J9kyd6slSZrJMFsArwD/\nrKreC6wDrk/yPuAGYE9VrQH2tHmAy4E17bYNuBV6gQHcCFwKXALceCI0JEnjN2MAVNXhqvpWm/6/\nwJPACmAzsLMN2wlc1aY3A7dXz/3AWUnOBy4DdlfVsao6DuwGNs3q2kiShjbSMYAkq4EPAg8Ay6vq\nMPRCAjivDVsBPNv3sIOtNl1dkjQBQwdAkr8A/Ffgk1X1o5MNHVCrk9Sn/pxtSfYl2Xf06NFh25Mk\njWioAEiyhN6b/+9X1R+28pG2a4d2/3yrHwRW9T18JXDoJPXXqartVbW2qtYuW7ZslHWRJI1gmLOA\nAtwGPFlV/65v0S7gxJk8W4C7++rXtrOB1gEvtl1E9wIbkyxtB383tpokaQKGuRTEeuAfAI8mebjV\n/iXwGeDOJFuBZ4Cr27J7gCuAA8BLwHUAVXUsyc3Ag23cTVV1bFbWQpI0shkDoKr+N4P33wNsGDC+\ngOunea4dwI5RGpQkzQ0vBidpoEFfFuMXxSwuXgpC0tC8uN/iYgBIUkcZAJLUUR4D0Kzwi1+khcct\nAM0K3/ylhccAkKSOMgAkqaMMAEnqKANAkjrKAJCkjjIAJKmjDABJ6igDQJI6yk8CSxqJVwldPNwC\nkHTa/CT4wmQASFJHGQCS1FEGgCR1lAeBNRIv+ywtHm4BaCS++UuLx4wBkGRHkueTPNZXOzvJ7iT7\n2/3SVk+SW5IcSPJIkov7HrOljd+fZMvcrI4kaVjDbAH8Z2DTlNoNwJ6qWgPsafMAlwNr2m0bcCv0\nAgO4EbgUuAS48URoSJImY8YAqKo/AY5NKW8GdrbpncBVffXbq+d+4Kwk5wOXAbur6lhVHQd288ZQ\nkSSN0akeA1heVYcB2v15rb4CeLZv3MFWm64uSZqQ2T4InAG1Okn9jU+QbEuyL8m+o0ePzmpzkqTX\nnOppoEeSnF9Vh9sunudb/SCwqm/cSuBQq394Sn3voCeuqu3AdoC1a9cODAlJ84/XCFp4TnULYBdw\n4kyeLcDdffVr29lA64AX2y6ie4GNSZa2g78bW03SIuZpw/PbjFsASb5E77/3c5McpHc2z2eAO5Ns\nBZ4Brm7D7wGuAA4ALwHXAVTVsSQ3Aw+2cTdV1dQDy5KkMZoxAKrqY9Ms2jBgbAHXT/M8O4AdI3Wn\nifETv9Li5yeBNZBv/tLiZwBIUkcZAJLUUQaAJHWUASBJHWUASFJH+YUwHefpnlJ3GQAd55u/5pqX\niJi/3AUkaez8x2N+MAAkqaMMAEnqKANAkjrKg8CSJsKDw5NnAHSEp3tqIfA1Ol7uAuoI/7AkTWUA\nSFJHGQCS1FEeA1hE3M+vxcCDw+PjFsAi4pu/Fitf23PDLQBJC4JbBrPPAFiA3NUj9fh3cHrGvgso\nyaYkTyU5kOSGcf/8xcAXvaTZMNYtgCRnAL8D/CpwEHgwya6qemKcfSwU/qcvzcxdQ6du3LuALgEO\nVNXTAEnuADYDnQ4A3+il2XXk5ZcHBsObgJ8NGN/VwBh3AKwAnu2bPwhcOuYe5tx0b+jTvfgkjcd0\nf3/TBcZiD4ZxB0AG1Op1A5JtwLY2++MkT815V6M5F/izU3ngBN78T7nXCbHfuWW/IzrC4DetaUy8\n3z4/N8ygcQfAQWBV3/xK4FD/gKraDmwfZ1OjSLKvqtZOuo9hLKRewX7nmv3OrYXWL4z/LKAHgTVJ\nLkzyFuAaYNeYe5AkMeYtgKp6JcmvA/cCZwA7qurxcfYgSeoZ+wfBquoe4J5x/9xZNG93Tw2wkHoF\n+51r9ju3Flq/pKpmHiVJWnS8GJwkdZQBMEWS9yR5uO/2oySfnDLmw0le7BvzrybVb+vnnyZ5PMlj\nSb6U5K1Tlp+Z5Mvt8hsPJFk9mU5f7Wemfj+e5Gjf7/cfTarX1s8nWq+PT30ttOVJckv7/T6S5OJJ\n9NnXz0z9TvT1m2RHkueTPNZXOzvJ7iT72/3SaR67pY3Zn2TLAuj3p32/5/l3wktVeZvmRu9A9XPA\nz02pfxj46qT7a72sAL4PvK3N3wl8fMqYfwL8Xpu+BvjyPO/348B/mPTvtvXyV4HHgLfTO2b2P4A1\nU8ZcAfwRvVPG1wEPzPN+J/r6BT4EXAw81lf7N8ANbfoG4LMDHnc28HS7X9qml87XftuyH0/q9zzM\nzS2Ak9sAfK+qfjDpRmbwZuBtSd5M7w//0JTlm4GdbfouYEOSET7fMutm6nc+eS9wf1W9VFWvAP8T\n+FtTxmwGbq+e+4Gzkpw/7kabYfqdqKr6E+DYlHL/a3QncNWAh14G7K6qY1V1HNgNbJqzRpvT6Hfe\nMwBO7hrgS9Ms+6Uk30nyR0neP86m+lXV/wH+LfAMcBh4sar+eMqwVy/B0d4UXgTOGWefJwzZL8Df\nabtT7kqyasDycXkM+FCSc5K8nd5/+1P7GXSJkxVj6m+qYfqFefL67bO8qg4DtPvzBoyZT7/nYfoF\neGuSfUnuTzLvQsIAmEb7oNqVwB8MWPwteruFPgD8e+C/jbO3fm3f42bgQuAvA+9I8venDhvw0Imc\n/jVkv/8dWF1Vf43eLoydTEhVPQl8lt5/m18DvgO8MmXYvPn9DtnvvHn9jmje/J5HcEH1Ph38d4HP\nJ/krk26onwEwvcuBb1XVkakLqupHVfXjNn0PsCTJueNusPkV4PtVdbSqXgb+EPjlKWNevQRH2+3y\nTt64STsuM/ZbVS9U1U/a7H8EfnHMPb5OVd1WVRdX1Yfo/d72Txky4yVOxmmmfufZ6/eEIyd2m7X7\n5weMmU+/52H6paoOtfungb3AB8fV4DAMgOl9jGl2/yR514l96Ekuofd7fGGMvfV7BliX5O2tpw3A\nk1PG7AJOnDHxUeDr1Y5QTcCM/U7Zf37l1OXjluS8dn8B8Ld54+tiF3BtOxtoHb3dWofH3OarZup3\nnr1+T+h/jW4B7h4w5l5gY5KlbUtyY6tNwoz9tj7PbNPnAuuZb5e+n/RR6Pl4o3dg8gXgnX21XwN+\nrU3/OvA4vc3r+4FfnnC/vwl8l97+3y8CZwI3AVe25W+ltyvrAPBN4Ofneb//uu/3+w3gFybc7/+i\n94f7HWDDgNdD6H3R0feAR4G187zfib5+6QXSYeBlev/Vb6V3TGoPva2VPcDZbexa4At9j/2H7XV8\nALhuPvdLb8v20fZ7fhTYOsnXxaCbnwSWpI5yF5AkdZQBIEkdZQBIUkcZAJLUUQaAJHWUASBJHWUA\nSFJHGQCS1FH/H7e1Hh+J5g39AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadff224b10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d.fit( data )\n",
    "d.plot( n=100000, edgecolor='c', color='c', bins=50 )\n",
    "print d.parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We generated data from a normal distribution with mean 9 and standard deviation 0.4, and we can see that the distribution object was able to recover it. This is not surprising given the amount of data it received.\n",
    "\n",
    "The fitting process discards the previous parameters, and uses a pure MLE estimate on the new data. In this case, it just takes the mean and variance of the new data, discarding its previous parameters. It does not matter if the previous distribution was `NormalDistribution(1, 5)` or `NormalDistribution(10000000, 10000000)`, they would both recover the same parameters after.\n",
    "\n",
    "A way of taking into account current parameters is by using inertia, where the update is equal to `previous parameters * inertia + new parameters * ( 1 - inertia )`. This allows the update to take into account prior parameters a bit. Lets see an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d -- mu: 9.00497466779 sigma 0.39447571955\n",
      "e -- mu: 104.502487334 sigma 0.697237859775\n"
     ]
    }
   ],
   "source": [
    "d = NormalDistribution( 10, 1 )\n",
    "e = NormalDistribution( 200, 1 )\n",
    "\n",
    "d.fit( data, inertia=0.0 ) # The default setting\n",
    "e.fit( data, inertia=0.5 )\n",
    "\n",
    "print \"d -- mu: {} sigma {}\".format( *d.parameters )\n",
    "print \"e -- mu: {} sigma {}\".format( *e.parameters )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Out of core updates are also supported for distributions. This means that we can update a distribution using an amount of data which cannot fit into memory normally, but still produce an exact update. This is done by summarizing the dataset into sufficient statistics at each step, and then producing an update at the end. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d -- mu: 9.00497466779 sigma 0.39447571955\n"
     ]
    }
   ],
   "source": [
    "d = NormalDistribution( 105, 12 )\n",
    "\n",
    "for i in range(10):\n",
    "    d.summarize( data[i::10] )\n",
    "\n",
    "d.from_summaries()\n",
    "print \"d -- mu: {} sigma {}\".format( *d.parameters )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this particular case, the data can all fit in memory, but it doesn't necessarily have to. You just have to call `summarize`, then load up the next set of data, and continue.\n",
    "\n",
    "If we want to weight points, we can easily pass in a weight vector. Weights must be non-negative, and the sum of the weight vector must be greater than 0. Numerical stability issues can arise if you have entirely small weight vectors, so try to avoid that if possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d -- mu: 9.00249539503 sigma 0.395023883825\n"
     ]
    }
   ],
   "source": [
    "d = NormalDistribution( 105, 12 )\n",
    "weights = np.random.exponential( 100, 10000 )\n",
    "d.fit( data, weights )\n",
    "\n",
    "print \"d -- mu: {} sigma {}\".format( *d.parameters )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It doesn't change much in this case since the data was drawn from a perfect normal distribution, but it illustrates the possibility. The analog for summarization is the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d -- mu: 9.00249539503 sigma 0.395023883826\n"
     ]
    }
   ],
   "source": [
    "d = NormalDistribution( 105, 12 )\n",
    "\n",
    "for i in range(10):\n",
    "    d.summarize( data[i::10], weights=weights[i::10] )\n",
    "\n",
    "d.from_summaries()\n",
    "print \"d -- mu: {} sigma {}\".format( *d.parameters )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also easily sample from distributions by calling the `sample` function. `plot` does this internally when we call it, but to be explicit lets reproduce that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([   3.,    4.,    7.,   20.,   57.,  114.,  202.,  362.,  574.,\n",
       "         786., 1033., 1135., 1265., 1229., 1061.,  819.,  539.,  356.,\n",
       "         246.,  110.,   39.,   29.,    5.,    3.,    2.]),\n",
       " array([ 58.28767007,  62.01813627,  65.74860246,  69.47906866,\n",
       "         73.20953485,  76.94000105,  80.67046725,  84.40093344,\n",
       "         88.13139964,  91.86186583,  95.59233203,  99.32279823,\n",
       "        103.05326442, 106.78373062, 110.51419681, 114.24466301,\n",
       "        117.97512921, 121.7055954 , 125.4360616 , 129.16652779,\n",
       "        132.89699399, 136.62746019, 140.35792638, 144.08839258,\n",
       "        147.81885877, 151.54932497]),\n",
       " <a list of 25 Patch objects>)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEe9JREFUeJzt3X+MnNdd7/H3h9hJSa+K82PjFNu9\nDtQCQgU0skLAEkI1lyahqvNHI1IhapVIFlL4WRBNqEQECIkKdHOpVIKsJsS9qpLmhkIsFGittFaF\nRdI6bUmTpiFLCvE2ibMoP/gRoDZ8+WOOb6b2rHe9s7vj3fN+SaN5nvOcmTlzdvb57DnPM8+mqpAk\n9edbJt0ASdJkGACS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkTq2bdANO5+KLL66t\nW7dOuhmStKo88sgj/1hVU/PVO6sDYOvWrRw+fHjSzZCkVSXJPyyknlNAktQpA0CSOmUASFKnDABJ\n6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUqbP6m8DS2eDSQ4c4euzYgutvXL+e53fsWMYWSUvDEYA0\njzPZ+S+mvjQpBoAkdcoAkKROGQCS1CkDQJI65VlA0jLIwYMLrutZQ5qUeUcASe5M8kKSx4bKfi/J\nV5M8muRPk2wY2nZLkukkTyZ5+1D51a1sOsnNS/9WpNXJs4Y0KQuZAroLuPqksgPAW6rq+4C/BW4B\nSHI5cAPwve0xf5jknCTnAB8GrgEuB97d6kqSJmTeAKiqzwIvnlT2qao63lYfAja35V3APVX1H1X1\nNWAauLLdpqvq6ar6BnBPqytJmpClOAj8M8BftOVNwJGhbTOtbK7yUyTZk+RwksOzs7NL0DxJ0ihj\nBUCSDwDHgY+dKBpRrU5Tfmph1d6q2l5V26em5v2n9pKkRVr0WUBJdgPvAHZW1Ymd+QywZajaZuDZ\ntjxXuSRpAhY1AkhyNfB+4J1V9erQpv3ADUnOS3IZsA34HPB5YFuSy5Kcy+BA8f7xmi5JGse8I4Ak\ndwM/ClycZAa4lcFZP+cBB5IAPFRVP1tVjye5F/gKg6mhm6rqP9vz/BzwSeAc4M6qenwZ3o8kaYHm\nDYCqeveI4jtOU/93gN8ZUf4A8MAZtU6StGy8FIQkdcoAkKROGQCS1CkDQJI6ZQBIUqe8HLS6c6b/\n5F1aqxwBqDvu/KUBA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhS\np7wWkHQWyMGDC667cf16nt+xY/kao244ApBWGa9lpKViAEhSpwwASeqUASBJnfIgsFY1/7mLtHjz\njgCS3JnkhSSPDZVdmORAkqfa/QWtPEk+lGQ6yaNJrhh6zO5W/6kku5fn7ag37vylxVvIFNBdwNUn\nld0MPFhV24AH2zrANcC2dtsD3A6DwABuBX4QuBK49URoSJImY94AqKrPAi+eVLwL2NeW9wHXDZV/\ntAYeAjYkeSPwduBAVb1YVS8BBzg1VCRJK2ixB4E3VtVzAO3+kla+CTgyVG+mlc1VLkmakKU+Cygj\nyuo05ac+QbInyeEkh2dnZ5e0cZKk1yw2AI62qR3a/QutfAbYMlRvM/DsacpPUVV7q2p7VW2fmppa\nZPMkSfNZbADsB06cybMbuH+o/D3tbKCrgFfaFNEngR9PckE7+PvjrUySNCHzfg8gyd3AjwIXJ5lh\ncDbP7wL3JrkReAa4vlV/ALgWmAZeBd4LUFUvJvlt4POt3m9V1ckHliVJK2jeAKiqd8+xaeeIugXc\nNMfz3AnceUatkyQtGy8FIUmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIA\nJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CS\nOmUASFKnDABJ6tRYAZDkl5M8nuSxJHcneV2Sy5I8nOSpJB9Pcm6re15bn27bty7FG5AkLc66xT4w\nySbgF4DLq+rfktwL3ABcC9xWVfck+SPgRuD2dv9SVb05yQ3AB4GfHPsdSB3KwYMLrrtx/Xqe37Fj\n+RqjVWvcKaB1wLcmWQecDzwHvA24r23fB1zXlne1ddr2nUky5utLmsfRY8cm3QSdpRYdAFX1deD3\ngWcY7PhfAR4BXq6q463aDLCpLW8CjrTHHm/1L1rs60uSxrPoAEhyAYO/6i8Dvh14PXDNiKp14iGn\n2Tb8vHuSHE5yeHZ2drHNkyTNY5wpoB8DvlZVs1V1DPgE8MPAhjYlBLAZeLYtzwBbANr2bwNePPlJ\nq2pvVW2vqu1TU1NjNE+SdDqLPgjMYOrnqiTnA/8G7AQOA58B3gXcA+wG7m/197f1v27bP11Vp4wA\n1LdLDx1yzlpaIeMcA3iYwcHcLwBfbs+1F3g/8L4k0wzm+O9oD7kDuKiVvw+4eYx2a41y5y+tnHFG\nAFTVrcCtJxU/DVw5ou6/A9eP83qSpKXjN4ElqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqU\nASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkA\nktQpA0CSOmUASFKnDABJ6pQBIEmdGisAkmxIcl+SryZ5IskPJbkwyYEkT7X7C1rdJPlQkukkjya5\nYmnegiRpMcYdAfwB8JdV9d3A9wNPADcDD1bVNuDBtg5wDbCt3fYAt4/52pKkMSw6AJK8AfgR4A6A\nqvpGVb0M7AL2tWr7gOva8i7gozXwELAhyRsX3XJJ0ljGGQF8BzAL/HGSLyb5SJLXAxur6jmAdn9J\nq78JODL0+JlWJkmagHECYB1wBXB7Vb0V+Fdem+4ZJSPK6pRKyZ4kh5Mcnp2dHaN5kqTTGScAZoCZ\nqnq4rd/HIBCOnpjaafcvDNXfMvT4zcCzJz9pVe2tqu1VtX1qamqM5kmSTmfRAVBVzwNHknxXK9oJ\nfAXYD+xuZbuB+9vyfuA97Wygq4BXTkwVSZJW3roxH//zwMeSnAs8DbyXQajcm+RG4Bng+lb3AeBa\nYBp4tdWVJE3IWAFQVV8Cto/YtHNE3QJuGuf1tPpceugQR48dm3QzJI3gN4G1rNz5S2cvA0CSOmUA\nSFKnDABJ6pQBIEmdGvc0UEmrQA4ePKP6G9ev5/kdO5anMTprOAKQdArP3uqDASBJnTIAJKlTBoAk\ndcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKn\nDABJ6pQBIEmdGjsAkpyT5ItJ/rytX5bk4SRPJfl4knNb+Xltfbpt3zrua0uSFm8pRgC/CDwxtP5B\n4Laq2ga8BNzYym8EXqqqNwO3tXqSpAkZKwCSbAZ+AvhIWw/wNuC+VmUfcF1b3tXWadt3tvqSpAkY\ndwTwf4BfA/6rrV8EvFxVx9v6DLCpLW8CjgC07a+0+t8kyZ4kh5Mcnp2dHbN5kqS5LDoAkrwDeKGq\nHhkuHlG1FrDttYKqvVW1vaq2T01NLbZ5kqR5rBvjsTuAdya5Fngd8AYGI4INSda1v/I3A8+2+jPA\nFmAmyTrg24AXx3h9SdIYFj0CqKpbqmpzVW0FbgA+XVU/BXwGeFerthu4vy3vb+u07Z+uqlNGAJKk\nlbEc3wN4P/C+JNMM5vjvaOV3ABe18vcBNy/Da0uSFmicKaD/r6oOAgfb8tPAlSPq/Dtw/VK8niRp\nfH4TWJI6tSQjAPXj0kOHOHrs2KSbIWkJGAA6I+78+5GDBxdcd+P69Ty/Y8fyNUbLwikgSWPzD4PV\nyQCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcM\nAEnqlAEgSZ0yACSpUwaAJHXK/wgmaUn4H8RWH0cAklac/0Hs7GAASFKnFh0ASbYk+UySJ5I8nuQX\nW/mFSQ4keardX9DKk+RDSaaTPJrkiqV6E5KkMzfOCOA48CtV9T3AVcBNSS4HbgYerKptwINtHeAa\nYFu77QFuH+O1JUljWnQAVNVzVfWFtvzPwBPAJmAXsK9V2wdc15Z3AR+tgYeADUneuOiWS5LGsiTH\nAJJsBd4KPAxsrKrnYBASwCWt2ibgyNDDZlqZJGkCxj4NNMn/AP4E+KWq+qckc1YdUVYjnm8Pgyki\n3vSmN43bPM3j0kOHPCND6tRYI4Ak6xns/D9WVZ9oxUdPTO20+xda+QywZejhm4FnT37OqtpbVdur\navvU1NQ4zdMCuPOX+jXOWUAB7gCeqKr/PbRpP7C7Le8G7h8qf087G+gq4JUTU0WSpJU3zhTQDuCn\ngS8n+VIr+3Xgd4F7k9wIPANc37Y9AFwLTAOvAu8d47UlSWNadABU1V8xel4fYOeI+gXctNjXkyQt\nLb8JLEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnfKfwq8xXt1T0kIZAGuM\nO3+tFjl4cMF1N65fz/M7dixfYzrlFJCks55/2CwPA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1\nyu8BnMX8Upek5eQI4Czmzl/ScjIAJKlTTgFJWhW8dMTScwQgac1x+nRhHAFIWpPOZMQAfY4aVnwE\nkOTqJE8mmU5y80q/viSN0uOoYUVHAEnOAT4M/C9gBvh8kv1V9ZWVbMekeFqnpLPJSo8ArgSmq+rp\nqvoGcA+wa4XbMDHu/CWdTVb6GMAm4MjQ+gzwg8v1Ymf6F/e3AP91Bs9/pvUlnd3O5LjBcu8vVuKY\nxEoHQEaU1TdVSPYAe9rqvyR5conbcDHwj6M2nOnOfJXv/Ofsh47YB/YBLLIPlnt/cZTRO8wF+p8L\nqbTSATADbBla3ww8O1yhqvYCe5erAUkOV9X25Xr+1cJ+sA/APoC++2CljwF8HtiW5LIk5wI3APtX\nuA2SJFZ4BFBVx5P8HPBJ4Bzgzqp6fCXbIEkaWPEvglXVA8ADK/26Q5ZtemmVsR/sA7APoOM+SFXN\nX0uStOZ4LSBJ6tSaD4AkG5Lcl+SrSZ5I8kNJLkxyIMlT7f6CSbdzOSX55SSPJ3ksyd1JXtcOxD/c\n+uDj7aD8mpHkziQvJHlsqGzkzz0DH2qXJ3k0yRWTa/nSmqMffq/9Pjya5E+TbBjadkvrhyeTvH0y\nrV5ao/pgaNuvJqkkF7f1NftZGGXNBwDwB8BfVtV3A98PPAHcDDxYVduAB9v6mpRkE/ALwPaqeguD\ng+83AB8Ebmt98BJw4+RauSzuAq4+qWyun/s1wLZ22wPcvkJtXAl3cWo/HADeUlXfB/wtcAtAkssZ\nfDa+tz3mD9vlW1a7uzi1D0iyhcFlaZ4ZKl7Ln4VTrOkASPIG4EeAOwCq6htV9TKDy0/sa9X2AddN\npoUrZh3wrUnWAecDzwFvA+5r29dcH1TVZ4EXTyqe6+e+C/hoDTwEbEjyxpVp6fIa1Q9V9amqOt5W\nH2LwfRwY9MM9VfUfVfU1YJrB5VtWtTk+CwC3Ab/GN38Zdc1+FkZZ0wEAfAcwC/xxki8m+UiS1wMb\nq+o5gHZ/ySQbuZyq6uvA7zP4K+c54BXgEeDloZ3ADIPLdKx1c/3cR12ipIf+APgZ4C/acjf9kOSd\nwNer6m9O2tRNH8DaD4B1wBXA7VX1VuBfWcPTPaO0ee5dwGXAtwOvZzDMPVnPp4PNe4mStSjJB4Dj\nwMdOFI2otub6Icn5wAeA3xi1eUTZmuuDE9Z6AMwAM1X1cFu/j0EgHD0xrGv3L0yofSvhx4CvVdVs\nVR0DPgH8MIOh7YnvgZxySY41aq6f+7yXKFlrkuwG3gH8VL12Lngv/fCdDP4g+pskf8/gfX4hyaX0\n0wfAGg+AqnoeOJLku1rRTuArDC4/sbuV7Qbun0DzVsozwFVJzk8SXuuDzwDvanXWeh+cMNfPfT/w\nnnYGyFXAKyemitaiJFcD7wfeWVWvDm3aD9yQ5LwklzE4EPq5SbRxOVXVl6vqkqraWlVbGez0r2j7\ni64+C1TVmr4BPwAcBh4F/gy4ALiIwVkgT7X7CyfdzmXug98Evgo8Bvxf4DwGx0c+x+BA3/8Dzpt0\nO5f4Pd/N4JjHMQa/4DfO9XNnMOz/MPB3wJcZnDE18fewjP0wzWCe+0vt9kdD9T/Q+uFJ4JpJt3+5\n+uCk7X8PXLzWPwujbn4TWJI6taangCRJczMAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnq\n1H8DiJ6+BJE62H0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadff0ec710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = NormalDistribution( 105, 12 )\n",
    "samples = [ d.sample() for i in range(10000) ]\n",
    "plt.hist( samples, edgecolor='c', color='c', bins=25 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Discrete Distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A notable deviant from the numerics logic is the Discrete Distribution. Instead of passing parameters as floats, you instead pass in a dictionary where keys can be ~any object~ and values are the probability of them occurring. If you try to calculate the log probability of an item not present in the distribution, the default behavior is to return negative infinity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-2.30258509299\n",
      "-0.69314718056\n",
      "-inf\n"
     ]
    }
   ],
   "source": [
    "d = DiscreteDistribution({'A': 0.1, 'C': 0.25, 'G': 0.50, 'T': 0.15})\n",
    "print d.log_probability( 'A' )\n",
    "print d.log_probability( 'G' )\n",
    "print d.log_probability( '????' )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Updating a discrete distributon is the same as updating other distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logp(A) = -0.826678573184\n",
      "logp(C) = -1.85629799037\n",
      "logp(G) = -1.56861591791\n",
      "logp(T) = -1.6199092123\n",
      "[{'A': 0.4375, 'C': 0.15625, 'T': 0.19791666666666666, 'G': 0.20833333333333334}]\n"
     ]
    }
   ],
   "source": [
    "items = list('ACGATACACTGAATGACAGCAGTCACTGACAGTAGTACGAGTAGTAGCAGAGAGTAATAAAGAATTAATATATGACACTACGAAAAAAATGCATCG')\n",
    "d.fit( items )\n",
    "\n",
    "for char in 'ACGT':\n",
    "    print \"logp({}) = {}\".format( char, d.log_probability( char ) )\n",
    "\n",
    "print d.parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also take a look at the JSON serialization. JSONs are the perfect way to store these models given their human interpretability, and the ability to recursively store JSONs within other JSONs, such as more complicated models (such as HMMs) will need to store the JSON of each distribution. It is useful to store these models after the computationally intensive task of training them, so that it need not be repeated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"frozen\" :false,\n",
      "    \"dtype\" :\"str\",\n",
      "    \"class\" :\"Distribution\",\n",
      "    \"parameters\" :[\n",
      "        {\n",
      "            \"A\" :0.4375,\n",
      "            \"C\" :0.15625,\n",
      "            \"T\" :0.19791666666666666,\n",
      "            \"G\" :0.20833333333333334\n",
      "        }\n",
      "    ],\n",
      "    \"name\" :\"DiscreteDistribution\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print d.to_json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Kernel Density"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Gaussian Kernel Density is a non-parametric distribution which stores a series of points, their associated weights, and the bandwidth representing the range of influence of each point. This will become apparent with an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAER5JREFUeJzt3W2MpWV9x/HvT0CtDw2QBdwua5c2\nm7bUViQToCVptsEiUOPaFySYVLeUZNsEW21sKmpSGn1D0larqSXZ6laMFEN8CJuGilsqMSXBsijy\nIFo2aGFk3V2LRVuSCvLvi7kHDrvzcObMw33Oub6fZDLnXHOfM//Znbl+53q475OqQpLUnhf1XYAk\nqR8GgCQ1ygCQpEYZAJLUKANAkhplAEhSowwASWqUASBJjTIAJKlRJ/ZdwFI2bdpU27Zt67sMSZoo\n99xzz/er6rTljhvrANi2bRsHDhzouwxJmihJ/nOY45wCkqRGGQCS1CgDQJIaZQBIUqMMAElqlAEg\nSY0yACSpUQaAJDXKAJCkRo31mcBqz6vuvJPDTz8NwBknncT3Lryw54qk6eUIQGNlvvM/9raktecI\nQGMtd9wBOBqQ1oMjAE0ERwPS2jMAJKlRTgFp4rhQLK0NA0ATY349YJBTQ9LoDAD1bvAVvaSN4xqA\nemfnL/XDEYAmnltFpdE4AtDUcCQhrYwBIEmNcgpIvXDhV+qfAaBerFfnP7hV1DUBaWlOAWlqOcKQ\nluYIQM3xTGJpjgGgDdPnvP9i39tRglrmFJA2TJ+drR29dDwDQJIaZQBIUqNcA9BUW+gKopLmOAKQ\npEY5ApDGiFtUtZEMAK0rL/mwMoP/Vv67ab0ZAFpXdmLP89W9xo0BIK2xxTp6X91r3BgA0hobtqP3\njWzUt2V3ASXZmuRLSR5K8mCSd3TtpybZn+Th7vMpXXuSfCTJwST3JTl34Ll2dcc/nGTX+v1Y0uRw\nNKC+DDMCeAZ4V1V9NckrgXuS7Ad+D7i9qq5Lcg1wDfBu4FJge/dxPnA9cH6SU4FrgRmguufZV1U/\nWOsfShrF4NTNi4Bnu3ZfoWtaLRsAVXUIONTd/lGSh4AtwE5gR3fYDcAdzAXATuCTVVXAXUlOTrK5\nO3Z/VT0B0IXIJcBNa/jzSCMbfCX+7CLt68UT1tSHFa0BJNkGvA74CnBGFw5U1aEkp3eHbQEeG3jY\nbNe2WLukFXA3kdbK0GcCJ3kF8FngnVX1w6UOXaCtlmg/9vvsTnIgyYGjR48OW57UDHcTaa0MNQJI\nchJznf+NVfW5rvlwks3dq//NwJGufRbYOvDwM4HHu/Ydx7Tfcez3qqo9wB6AmZmZ4wJCWmtrMf2y\nXie8uVNI62mYXUABPg48VFUfHPjSPmB+J88u4JaB9rd1u4EuAJ7spopuAy5Ockq3Y+jirk2aeMtt\n91xtyPhKX+thmBHAhcBbgfuT3Nu1vRe4Drg5yVXAo8Dl3dduBS4DDgJPAVcCVNUTST4A3N0d9/75\nBWFJ0sYbZhfQv7Hw/D3ARQscX8DVizzXXmDvSgqUxpXXOdKk80xgrYkWd6bY+WvSGQBaE63sTPFV\nv6aJAaB1Ma0nNtn5a5r4jmCS1ChHANKEmNZRlfpjAEhDsPPVNDIApAnn2cIalWsA0pRwgVorZQBI\nUqOcApKmiNNBWglHANIUcjpIwzAAJKlRBoAkNco1AGlKuR6g5TgCkKac6wFajAEgSY0yACSpUQaA\nJDXKRWCNzDdHkSabIwCNzM5fmmwGgCQ1ygCQpEa5BiA1ZnDtxpPE2mYASA1Y7B3NXMdpm1NAktQo\nA0CSGmUASFKjXAPQinjylzQ9HAFoRez8pelhAEhSowwASWqUawBS43znsHY5ApAEuL7TIgNAkhpl\nAEhSo5YNgCR7kxxJ8sBA218k+W6Se7uPywa+9p4kB5N8K8kbBtov6doOJrlm7X8USdJKDDMC+ARw\nyQLtH6qqc7qPWwGSnA1cAfxy95i/S3JCkhOAjwKXAmcDb+mOlST1ZNldQFX15STbhny+ncCnq+r/\ngG8nOQic133tYFU9ApDk092x31hxxZKkNbGaNYC3J7mvmyI6pWvbAjw2cMxs17ZYuySpJ6MGwPXA\nzwPnAIeAv+7as8CxtUT7cZLsTnIgyYGjR4+OWJ4kaTkjBUBVHa6qn1TVs8Df8/w0zyywdeDQM4HH\nl2hf6Ln3VNVMVc2cdtppo5QnSRrCSAGQZPPA3d8B5ncI7QOuSPKSJGcB24F/B+4Gtic5K8mLmVso\n3jd62ZKk1Vp2ETjJTcAOYFOSWeBaYEeSc5ibxvkO8AcAVfVgkpuZW9x9Bri6qn7SPc/bgduAE4C9\nVfXgmv80klbFy0K0JVULTsWPhZmZmTpw4EDfZTTP9wBoU+3Y0XcJGlGSe6pqZrnjPBNYy7Lzl6aT\nASBJjTIAJKlRBoAkNcoAkKRG+Y5gkhbkltDp5whA0pLcBTa9DABJapQBIEmNMgAkqVEGgCQ1ygCQ\npEYZAJLUKM8DkLQszwmYTgaAFuQloLUQfyemi1NAWpB/6NL0MwAkqVEGgCQ1yjUASSvigvD0cAQg\naSSuE00+A0CSGmUASFKjDABJapQBIEmNMgAkqVEGgCQ1ygCQpEYZAJLUKANAkhrlpSD0HC8BLbXF\nEYCeY+cvtcUAkKRGGQCS1CgDQJIa5SKwpJH53gCTbdkRQJK9SY4keWCg7dQk+5M83H0+pWtPko8k\nOZjkviTnDjxmV3f8w0l2rc+PI6kPbiCYTMNMAX0CuOSYtmuA26tqO3B7dx/gUmB797EbuB7mAgO4\nFjgfOA+4dj40JEn9WDYAqurLwBPHNO8Ebuhu3wC8eaD9kzXnLuDkJJuBNwD7q+qJqvoBsJ/jQ0WS\ntIFGXQQ+o6oOAXSfT+/atwCPDRw327Ut1i5J6sla7wLKAm21RPvxT5DsTnIgyYGjR4+uaXGSpOeN\nGgCHu6kdus9HuvZZYOvAcWcCjy/Rfpyq2lNVM1U1c9ppp41YniRpOaMGwD5gfifPLuCWgfa3dbuB\nLgCe7KaIbgMuTnJKt/h7cdemnr3qzjvJHXc8t51PUjuWPQ8gyU3ADmBTklnmdvNcB9yc5CrgUeDy\n7vBbgcuAg8BTwJUAVfVEkg8Ad3fHvb+qjl1YVg/cvqe14jkBk2fZAKiqtyzypYsWOLaAqxd5nr3A\n3hVVJ2ni+KJicngpCElqlAEgSY0yACSpUQaAJDXKAJCkRhkAktQoA0CSGmUASFKjDABJapRvCdmg\nV915p2drSnIE0CI7f0ngCEDSOvDCcJPBEYCkdeNoc7wZAJLUKANAkhplAEhSowwASWqUASBJjXIb\naCM8+UvSsRwBNMLOX9KxHAFIWleeFDa+HAFI2hCOQsePASBJjTIAJKlRBoAkNcoAkKRGGQCS1Ci3\ngUraMG4JHS+OACRtOLeEjgcDQJIa5RTQFPP6P5KW4ghgitn5S1qKASBJjTIAJKlRrgFI6oVbQvu3\nqhFAku8kuT/JvUkOdG2nJtmf5OHu8ylde5J8JMnBJPclOXctfgBJk821qv6sxRTQb1bVOVU1092/\nBri9qrYDt3f3AS4Ftncfu4Hr1+B7S5JGtB5rADuBG7rbNwBvHmj/ZM25Czg5yeZ1+P6SpCGsNgAK\n+GKSe5Ls7trOqKpDAN3n07v2LcBjA4+d7dokST1Y7SLwhVX1eJLTgf1JvrnEsVmgrY47aC5IdgO8\n+tWvXmV5kqTFrCoAqurx7vORJJ8HzgMOJ9lcVYe6KZ4j3eGzwNaBh58JPL7Ac+4B9gDMzMwcFxBa\nmmf/ahK5I6gfI08BJXl5klfO3wYuBh4A9gG7usN2Abd0t/cBb+t2A10APDk/VaS1Y+evSebv78Za\nzQjgDODzSeaf5x+r6gtJ7gZuTnIV8ChweXf8rcBlwEHgKeDKVXxvSdIqjRwAVfUI8NoF2v8LuGiB\n9gKuHvX7SZLWlpeCkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY0yACSpUb4hzAQavNyDp85LGpUB\nMIEGT5c//PTTz11HRZoGXhdo4zgFJGkseV2g9WcASFKjnAKSNLacDlpfjgAkjT2ng9aHASBJjTIA\nJKlRBoAkNcpF4Anhe/2qdS4Irz1HABPCzl+a49/C2jEAJKlRBoAkNcoAkKRGGQCS1Ch3AY0xd/5I\nC3NH0NpwBDDG7Pylpfk3sjqOACRNNEcDo3MEIGkqOBpYOUcAY8Z5f0kbxQAYM3b+0uicDloZp4Ak\nTR1fSA3HEcAYcNpHUh8cAYwBO39JfTAAJKlRTgH1xGkfaX3NLwi/CHi2a3Nx+IUcAfTEzl/aGM8O\n3Pbv7oUcAWwgX/VL/XOr6PM2PACSXAJ8GDgB+FhVXbfRNWwkO31pPB1++unnwgDaDIQNDYAkJwAf\nBX4LmAXuTrKvqr6xkXWsNzt9afIMBkIrYbDRI4DzgINV9QhAkk8DO4GJDIDBjn5woUnSZGvlBdxG\nB8AW4LGB+7PA+Rtcwwss1okPvgIYpqO385emy0K7iIa5PUmjh40OgCzQVi84INkN7O7u/k+Sbw18\neRPw/XWq7YW7BVi42FV09Ota+zqa1LrB2vswqXXDIrU/u8Lbi/Ud62ihun92mAdudADMAlsH7p8J\nPD54QFXtAfYs9OAkB6pqZv3KWz+TWvuk1g3W3odJrRsmt/bV1L3R5wHcDWxPclaSFwNXAPs2uAZJ\nEhs8AqiqZ5K8HbiNuW2ge6vqwY2sQZI0Z8PPA6iqW4FbR3z4glNDE2JSa5/UusHa+zCpdcPk1j5y\n3amq5Y+SJE0drwUkSY2a2ABI8qdJKsmmvmsZVpK/TPLNJPcl+XySk/uuaSlJLknyrSQHk1zTdz3D\nSrI1yZeSPJTkwSTv6LumlUhyQpKvJfmnvmtZiSQnJ/lM9zv+UJJf67umYST5k+735IEkNyV5ad81\nLSbJ3iRHkjww0HZqkv1JHu4+nzLs801kACTZytzlJB7tu5YV2g+8pqp+FfgP4D0917Oogct2XAqc\nDbwlydn9VjW0Z4B3VdUvARcAV09Q7QDvAB7qu4gRfBj4QlX9IvBaJuBnSLIF+GNgpqpew9zmlCv6\nrWpJnwAuOabtGuD2qtoO3N7dH8pEBgDwIeDPOOYksnFXVV+sqme6u3cxdx7EuHrush1V9WNg/rId\nY6+qDlXVV7vbP2KuI9rSb1XDSXIm8NvAx/quZSWS/DTwG8DHAarqx1X13/1WNbQTgZ9KciLwMo45\nN2mcVNWXgSeOad4J3NDdvgF487DPN3EBkORNwHer6ut917JKvw/8c99FLGGhy3ZMRCc6KMk24HXA\nV/qtZGh/w9yLm0m7usjPAUeBf+imrz6W5OV9F7Wcqvou8FfMzSYcAp6sqi/2W9WKnVFVh2DuxQ9w\n+rAPHMsASPIv3XzcsR87gfcBf953jYtZpvb5Y97H3DTFjf1VuqxlL9sx7pK8Avgs8M6q+mHf9Swn\nyRuBI1V1T9+1jOBE4Fzg+qp6HfC/rGAqoi/dfPlO4CzgZ4CXJ/ndfqvaOGP5hjBV9fqF2pP8CnP/\nUV9PAnNTKF9Ncl5VfW8DS1zUYrXPS7ILeCNwUY33HtxlL9sxzpKcxFznf2NVfa7veoZ0IfCmJJcB\nLwV+OsmnqmoSOqRZYLaq5kdan2ECAgB4PfDtqjoKkORzwK8Dn+q1qpU5nGRzVR1Kshk4MuwDx3IE\nsJiqur+qTq+qbVW1jblfunPHpfNfTvdmOO8G3lRVT/VdzzIm9rIdmXt18HHgoar6YN/1DKuq3lNV\nZ3a/21cA/zohnT/d3+BjSX6ha7qIybjM+6PABUle1v3eXMQELF4fYx+wq7u9C7hl2AeO5Qhgiv0t\n8BJgfzeCuauq/rDfkhY24ZftuBB4K3B/knu7tvd2Z6Fr/fwRcGP3guER4Mqe61lWVX0lyWeArzI3\nLfs1xviM4CQ3ATuATUlmgWuB64Cbk1zFXKBdPvTzjfcshCRpvUzUFJAkae0YAJLUKANAkhplAEhS\nowwASWqUASBJjTIAJKlRBoAkNer/AY4SsnJMFhNIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadff224410>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = GaussianKernelDensity( [0.5, 1.2, 4.2, 1.9, 5.4, 3.4], bandwidth=1 )\n",
    "d.plot( n=100000, edgecolor='c', color='c', bins=100 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Changing the bandwidth parameter is equivalent to reducing the variance on a Gaussian. It makes the densities more central around the points, but can cause overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEKhJREFUeJzt3X+MZWV9x/H3R8T+QA2YXWAD2LXN\npqltUiQbpCExtLS4UFPsHyaSVDfEZPsHNpo2adF/aDUm/lPbmliSLWxdUoUQlbgxG3FD3Vg3QVks\n5Ydo2VAq013ZtbQqNamo3/4xZ+xld2bnzsyde+65z/uVTObeZ87MfebOvc/nfJ/znDOpKiRJ7XlZ\n3x2QJPXDAJCkRhkAktQoA0CSGmUASFKjDABJapQBIEmNMgAkqVEGgCQ16uV9d+BstmzZUtu3b++7\nG5I0KA8//PB3qmrratvNdABs376do0eP9t0NSRqUJP8+znZOAUlSowwASWqUASBJjTIAJKlRBoAk\nNcoAkKRGGQCS1CgDQJIaZQBIUqNm+kxgSf27+MgRnnvxRQAuOvdcvn311T33SJNiAKgXDirDsfR3\nOv22hs8AaNAsDL4OKlL/DIAGOfjOrlkI51aMPtfQ5vNtAGhqTn/DadFKz4vP1eY6/flt8fl2FZCm\npsU32Dh8XtQXA0CSGuUUUCOcfpF0OiuARjj4SzqdFYAkdXL4MNDOiiADQNIZWp8ybOV3NwCmyDXe\nGopWBsDWeQxgijwBS9IsWTUAklyW5ItJnkzyRJL3dO2vSXIoyVPd5wu69iT5aJJjSR5NcsXIz9rd\nbf9Ukt2b92sNQw4fJocPc/GRI313RVKDxqkAfgT8SVX9CnAVcEuS1wO3Ag9U1Q7gge4+wPXAju5j\nD3A7LAYGcBvwRuBK4Lal0Gid1YBW4k6CNtOqAVBVJ6rqa93t7wNPApcANwL7u832A2/tbt8I3FWL\nHgTOT7INeDNwqKqer6r/Ag4Buyb620hzyp0EbYY1HQNIsh14A/AV4KKqOgGLIQFc2G12CfDsyLct\ndG0rtUuSejD2KqAkrwQ+Dby3qr6XZMVNl2mrs7Sf/jh7WJw64rWvfe243dMGtLb2WdKisQIgybks\nDv6fqKrPdM3PJdlWVSe6KZ6TXfsCcNnIt18KHO/arzmt/fDpj1VVe4G9ADt37jwjIIZmSOuph9JP\nSZMxziqgAHcCT1bVR0a+dABYWsmzG/jsSPs7u9VAVwHf7aaI7geuS3JBd/D3uq5trjmoSppV41QA\nVwPvAB5L8kjX9n7gw8C9Sd4FfAt4W/e1g8ANwDHgB8DNAFX1fJIPAg91232gqp6fyG8hSVqzVQOg\nqr7M8vP3ANcus30Bt6zws/YB+9bSQUmzZcjHjIY0JTsNngksaV2GOJAOsc+byQCQpEYZAJLUKK8G\nqpkw5HllaaisADRTnKOVpscAkKRGGQCS1CiPAUjSMlo4LmUFIElnMc/HpawA5tgsnPU4C32YRT4v\nmgVWAHNsFgaYWejDLPJ50SwwACSpUQaAJDXKAJCkRhkAktQoA0CSGuUyUEmAS1NbZAUgCXBpaous\nAGZEC6edS5otVgAzxr0wSdNiBbAJnEuVNARWAJvAwV/SEFgB6CU8FiG1wwpAy7KKkeafASBJjTIA\nJKlRHgOQNNdclbcyKwBJc83Bf2UGgCQ1ygCQpEZ5DEDSunneyLBZAUjaMOfZh8kKQBPnqgtpGKwA\nNHEO/tIwWAFIA+F8uybNCkAaGCssTYoBIEmNMgAkqVEGgCQ1atUASLIvyckkj4+0/XmS/0jySPdx\nw8jX3pfkWJJvJnnzSPuuru1Yklsn/6tIs+3iI0fI4cM/PZgr9W2cCuDjwK5l2v+qqi7vPg4CJHk9\n8HbgV7vv+dsk5yQ5B/gYcD3weuCmblupGR681axZdRloVX0pyfYxf96NwD1V9b/AvyU5BlzZfe1Y\nVT0NkOSebtuvr7nHOqt5OAnL5Y7SdGzkGMC7kzzaTRFd0LVdAjw7ss1C17ZS+xmS7ElyNMnRU6dO\nbaB7bRr64D9qnn4XDdvS1N3FR4703ZWJWm8A3A78EnA5cAL4y649y2xbZ2k/s7Fqb1XtrKqdW7du\nXWf3JGny5m2nZF1nAlfVc0u3k/wd8Lnu7gJw2cimlwLHu9srtes0ToFImoZ1VQBJto3c/X1gaYXQ\nAeDtSX4myeuAHcBXgYeAHUlel+QVLB4oPrD+brdh3vY2JM2WVSuAJHcD1wBbkiwAtwHXJLmcxWmc\nZ4A/BKiqJ5Lcy+LB3R8Bt1TVj7uf827gfuAcYF9VPTHx30aSNLZxVgHdtEzznWfZ/kPAh5ZpPwgc\nXFPvJEmbxjOBJalRBoAkNcr/B6AVuRpJmm9WAFqVq5Gk+WQFMCHzcAkGtcfXbdusACbEN5GGyNdt\n2wwASWqUASBJjTIAJKlRBoAkNcoAkKRGGQCS1CgDQJIaZQBIUqMMAElqlAEgSY3yWkCSJsKrxw6P\nFYCkifL6QsNhBaCJ8KqS0vBYAWgiHPyl4TEAJKlRBoAkNcpjAJLmjsekxmMFIGnuOPiPxwpg4NzT\naZNr7jUJVgAD5+DfNv/+2ggrgBnnnp6kzWIFMBDu6UmaNANAkhplAEhSowwASWqUASBJjXIVkGba\n0FdBeZ7G/Bn6a3KUFYAGYaiD6FD7rdXNw9/WAJCkRhkAktQoA0CSGmUASFKjVg2AJPuSnEzy+Ejb\na5IcSvJU9/mCrj1JPprkWJJHk1wx8j27u+2fSrJ7c34dSdK4xqkAPg7sOq3tVuCBqtoBPNDdB7ge\n2NF97AFuh8XAAG4D3ghcCdy2FBqSpH6seh5AVX0pyfbTmm8Erulu7wcOA3/Wtd9VVQU8mOT8JNu6\nbQ9V1fMASQ6xGCp3b/g3UG9c4z5M/t20ZL0ngl1UVScAqupEkgu79kuAZ0e2W+jaVmrXQCx38ouD\nyDD5d9OSSR8EzjJtdZb2M39AsifJ0SRHT506NdHOaeMcPKT5sd4AeK6b2qH7fLJrXwAuG9nuUuD4\nWdrPUFV7q2pnVe3cunXrOrunzZTDh39aEUgarvUGwAFgaSXPbuCzI+3v7FYDXQV8t5squh+4LskF\n3cHf67o2SVJPVj0GkORuFg/ibkmywOJqng8D9yZ5F/At4G3d5geBG4BjwA+AmwGq6vkkHwQe6rb7\nwNIBYUlSP8ZZBXTTCl+6dpltC7hlhZ+zD9i3pt7NuGmvpliadnkZ8JOpPaqkeeWZwBvQ1wFRB39J\nk2AASFKjDABJapQBIEmNMgAkqVEGgCQ1yn8KL2ni5ukfp88zKwBJm8ZrR802KwBp4Nzb1npZAUhz\nwr1trZUBIEmNMgAkqVEGgCQ1yoPAGgwPdups/F/Ha2cFoMHxTa7l+LpYOwNAkhplAEhSowwASWqU\nASBJjTIAJKlRBoAkNcoAkKRGGQCS1CjPBJbmnGfIaiVWANKcc/DXSqwApAnrc4/b6yVpLawApAmb\nhT3uWeiDZp8VgCSt09ArLisASdqgoVZcBoAkNcoAkKRGeQxgjVxTLWleWAGskYO/pHlhAEhSo5wC\nkubU0hJFaSUGgKRNNfS18vPMKSBJU+Hxs9ljAEhSowwASWrUhgIgyTNJHkvySJKjXdtrkhxK8lT3\n+YKuPUk+muRYkkeTXDGJX0CStD6TqAB+s6our6qd3f1bgQeqagfwQHcf4HpgR/exB7h9Ao8tSVqn\nzZgCuhHY393eD7x1pP2uWvQgcH6SbZvw+JKkMWw0AAr4QpKHk+zp2i6qqhMA3ecLu/ZLgGdHvneh\na3uJJHuSHE1y9NSpUxvsniRpJRs9D+Dqqjqe5ELgUJJvnGXbLNNWZzRU7QX2AuzcufOMr0uSJmND\nAVBVx7vPJ5PcB1wJPJdkW1Wd6KZ4TnabLwCXjXz7pcDxjTy+pLZ5ccaNWfcUUJLzkrxq6TZwHfA4\ncADY3W22G/hsd/sA8M5uNdBVwHeXpookaT0c/DdmIxXARcB9SZZ+zier6vNJHgLuTfIu4FvA27rt\nDwI3AMeAHwA3b+CxJUkbtO4AqKqngV9fpv0/gWuXaS/glvU+njTLnIrQEHkmsDQBDv4aIq8GqkHy\nCpPSxlkBaNDc85bWzwCQpEYZAJLUKI8BSNIEDPG4lBWAJE3QkI5LGQCS1CgDQJIaZQBIUqMMAElq\nlKuAJE3NEFfKzDMDYAxe6EuaLN9Ps8EpoDH4YpU0jwwASWqUASBJjfIYgKRB8Zjc5BgAGjxXlrTF\nwX9ynALS3HBgkNbGCkBaJ6citJKhVKVWANI6OfhrNbP+GjEAJKlRBoAkNcoAkKRGGQCS1CgDQJIa\n5TLQFbjET9K8swJYgYO/pHlnBSBp5lmRbw4DQFIv1nK2rIP/5jAANFeWBhXYnNPw3ROdPJ/P/ngM\nQHNrMwYWByvNEysASdpEs3xhOCsASZqCWawerQBGOL8r9WO5vWTfj5vPCmCELzapX6PvQd+Pm88A\nkKRGOQWkuTbLB+Ckvk09AJLsAv4GOAe4o6o+PO0+qD0bmU5wLnq6Rs/lmDeztkMy1QBIcg7wMeB3\ngAXgoSQHqurr0+zHKN/cWo2vD03acy++OBNhMO0K4ErgWFU9DZDkHuBGYKoB4KDfprW84XyNaFr6\nfJ1NOwAuAZ4dub8AvHHKffCN3biV9r4c9NWXvqqBaQdAlmmrl2yQ7AH2dHdfSPLNTe/VmbYA3+nh\ncSfBvq/Bcyz/olwHn/fpG2q/YYW+T/D1+AvjbDTtAFgALhu5fylwfHSDqtoL7J1mp06X5GhV7eyz\nD+tl3/th36dvqP2G2en7tM8DeAjYkeR1SV4BvB04MOU+SJKYcgVQVT9K8m7gfhaXge6rqiem2QdJ\n0qKpnwdQVQeBg9N+3DXqdQpqg+x7P+z79A213zAjfU9Vrb6VJGnueC0gSWqUAXCaJLuSfDPJsSS3\n9t2fcSXZl+Rkksf77staJLksyReTPJnkiSTv6btP40rys0m+muRfur7/Rd99Wqsk5yT55ySf67sv\na5HkmSSPJXkkydG++7MWSc5P8qkk3+he97/RW1+cAvp/3aUq/pWRS1UAN/V5qYpxJXkT8AJwV1X9\nWt/9GVeSbcC2qvpaklcBDwNvHchzHuC8qnohybnAl4H3VNWDPXdtbEn+GNgJvLqq3tJ3f8aV5Blg\nZ1UN7jyAJPuBf6qqO7rVkD9fVf/dR1+sAF7qp5eqqKofAkuXqph5VfUl4Pm++7FWVXWiqr7W3f4+\n8CSLZ4zPvFr0Qnf33O5jMHtUSS4Ffhe4o+++tCLJq4E3AXcCVNUP+xr8wQA43XKXqhjEYDQPkmwH\n3gB8pd+ejK+bQnkEOAkcqqrB9B34a+BPgZ/03ZF1KOALSR7urh4wFL8InAL+vpt6uyPJeX11xgB4\nqVUvVaHNkeSVwKeB91bV9/ruz7iq6sdVdTmLZ7VfmWQQ029J3gKcrKqH++7LOl1dVVcA1wO3dFOg\nQ/By4Arg9qp6A/A/QG/HGg2Al1r1UhWavG7+/NPAJ6rqM333Zz26Mv4wsKvnrozrauD3urn0e4Df\nSvIP/XZpfFV1vPt8EriPxenbIVgAFkYqxU+xGAi9MABeyktVTFl3IPVO4Mmq+kjf/VmLJFuTnN/d\n/jngt4Fv9Nur8VTV+6rq0qrazuLr/B+r6g967tZYkpzXLRigmz65DhjE6req+jbwbJJf7pquZcqX\nwx/lv4QcMeRLVSS5G7gG2JJkAbitqu7st1djuRp4B/BYN5cO8P7ujPFZtw3Y360eexlwb1UNajnl\nQF0E3Le478DLgU9W1ef77dKa/BHwiW4n82ng5r464jJQSWqUU0CS1CgDQJIaZQBIUqMMAElqlAEg\nSY0yACSpUQaAJDXKAJCkRv0fRBTQklmqOfIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadfef01d50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = GaussianKernelDensity( [0.5, 1.2, 4.2, 1.9, 5.4, 3.4], bandwidth=0.2 )\n",
    "d.plot( n=100000, edgecolor='c', color='c', bins=100 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can easily see the influence of weights by weighting a single point. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEsJJREFUeJzt3WGsXGd95/HvL4kpXVqa0BjHssM6\nVa2qdKVCZIWsLFUs6SaGoiYvGinVtlgokt9kV1S7Uhf6JioUib4pXaQtkpV413Rp0wiKEqGo1Apc\ndWsJiAMpEAIbb8qSKyexWYe0WVSawH9f3Mcwdu71nbm+d2bOPN+PdHXPeebM3P+xz5zfeZ5z5kyq\nCklSfy6bdQGSpNkwACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdumLWBVzM1Vdf\nXXv27Jl1GZI0KI8++uh3qmr7esvNdQDs2bOHEydOzLoMSRqUJP9nnOUcApKkThkAktQpA0CSOmUA\nSFKnxgqAJN9K8tUkjyU50dpel+RYkifb76tae5J8JMnJJF9Jcv3I6xxsyz+Z5ODWrJIkaRyT9AD+\nTVW9qar2tfn3Ag9X1V7g4TYP8HZgb/s5BHwUVgIDuBt4C3ADcPe50JAkTd+lDAHdChxt00eB20ba\nP1YrPg9cmWQncAtwrKrOVtXzwDHgwCX8fUnSJRg3AAr46ySPJjnU2nZU1TMA7ffrW/su4OmR5y63\ntrXaJUkzMO4HwfZX1akkrweOJfnGRZbNKm11kfbzn7wSMIcA3vCGN4xZniRpUmMFQFWdar9PJ/kU\nK2P4zyXZWVXPtCGe023xZeDakafvBk619rde0L60yt86DBwG2Ldvn99Yv0CuOX6c5156CYAd27bx\n7P79M65I6tu6Q0BJXpPkp89NAzcDXwMeBM5dyXMQeKBNPwi8q10NdCPwQhsi+gxwc5Kr2snfm1ub\nOnFu53/htKTZGKcHsAP4VJJzy/9ZVf1VkkeA+5PcCXwbuL0t/xDwDuAk8D3g3QBVdTbJB4BH2nLv\nr6qzm7YmkqSJrBsAVfUU8MurtP9f4KZV2gu4a43XOgIcmbxMSdJm85PAktQpA0CSOmUASFKn5voL\nYbTYsrQEeEmoNCv2ADRzXhIqzYYBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqU\nASBJnfJWENpSo98CJmm+2APQlnLnL80vA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4Z\nAJLUKQNAkjplAEhSp7wXkOZClpYA2LFtG8/u3z/bYqRO2APQXPHeQdL0GACS1CkDQJI6ZQBIUqcM\nAEnqlAEgSZ0aOwCSXJ7ky0k+3eavS/KFJE8m+Yskr2rtP9HmT7bH94y8xvta+zeT3LLZKyNJGt8k\nPYD3AE+MzP8h8OGq2gs8D9zZ2u8Enq+qnwc+3JYjyRuBO4BfAg4Af5Lk8ksrX5K0UWMFQJLdwK8B\n97T5AG8DPtEWOQrc1qZvbfO0x29qy98K3FdV36+qvwdOAjdsxkpIkiY3bg/gj4HfBX7Y5n8W+G5V\nvdzml4FdbXoX8DRAe/yFtvyP2ld5zo8kOZTkRJITZ86cmWBVJEmTWDcAkrwTOF1Vj442r7JorfPY\nxZ7z44aqw1W1r6r2bd++fb3yJEkbNM69gPYDv57kHcCrgdey0iO4MskV7Sh/N3CqLb8MXAssJ7kC\n+Bng7Ej7OaPPkSRN2bo9gKp6X1Xtrqo9rJzE/WxV/Tvgc8BvtMUOAg+06QfbPO3xz1ZVtfY72lVC\n1wF7gS9u2ppIkiZyKXcD/c/AfUn+APgycG9rvxf40yQnWTnyvwOgqh5Pcj/wdeBl4K6q+sEl/H1J\n0iWYKACqaglYatNPscpVPFX1T8Dtazz/g8AHJy1SkrT5/CSwJHXKAJCkThkAktQpA0CSOmUASFKn\nDABJ6pQBIEmdMgAkqVMGgCR16lJuBSGt6prjx3nupZdmXYakddgD0KZz5y8NgwEgSZ0yACSpUwaA\nJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE55N1DNnSwtAbBj2zae\n3b9/tsVIC8wegOaWdxWVtpYBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSp9YNgCSvTvLF\nJH+X5PEkv9/ar0vyhSRPJvmLJK9q7T/R5k+2x/eMvNb7Wvs3k9yyVSslSVrfOD2A7wNvq6pfBt4E\nHEhyI/CHwIerai/wPHBnW/5O4Pmq+nngw205krwRuAP4JeAA8CdJLt/MlZEkjW/dAKgVL7bZbe2n\ngLcBn2jtR4Hb2vStbZ72+E1J0trvq6rvV9XfAyeBGzZlLSRJExvrHECSy5M8BpwGjgH/G/huVb3c\nFlkGdrXpXcDTAO3xF4CfHW1f5TmSpCkbKwCq6gdV9SZgNytH7b+42mLtd9Z4bK328yQ5lOREkhNn\nzpwZpzxJ0gZMdBVQVX0XWAJuBK5Mcu5uoruBU216GbgWoD3+M8DZ0fZVnjP6Nw5X1b6q2rd9+/ZJ\nypMkTWCcq4C2J7myTf8k8KvAE8DngN9oix0EHmjTD7Z52uOfrapq7Xe0q4SuA/YCX9ysFZEkTWac\n7wPYCRxtV+xcBtxfVZ9O8nXgviR/AHwZuLctfy/wp0lOsnLkfwdAVT2e5H7g68DLwF1V9YPNXR1J\n0rjWDYCq+grw5lXan2KVq3iq6p+A29d4rQ8CH5y8TEnSZvOTwJLUKQNAkjplAEhSp/xSeG2Ka44f\n9zt8pYGxB6BN4c5fGh4DQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUA\nSFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSp/xOYM21LC0BsGPbNp7dv3+2\nxUgLxh6ABsHvHJY2nwEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6tS6AZDk2iSf\nS/JEkseTvKe1vy7JsSRPtt9XtfYk+UiSk0m+kuT6kdc62JZ/MsnBrVstSdJ6xukBvAz8p6r6ReBG\n4K4kbwTeCzxcVXuBh9s8wNuBve3nEPBRWAkM4G7gLcANwN3nQkOSNH3rBkBVPVNVX2rT/wg8AewC\nbgWOtsWOAre16VuBj9WKzwNXJtkJ3AIcq6qzVfU8cAw4sKlrI0ka20TnAJLsAd4MfAHYUVXPwEpI\nAK9vi+0Cnh552nJrW6v9wr9xKMmJJCfOnDkzSXmSpAmMHQBJfgr4JPA7VfUPF1t0lba6SPv5DVWH\nq2pfVe3bvn37uOVJkiY01u2gk2xjZef/8ar6y9b8XJKdVfVMG+I53dqXgWtHnr4bONXa33pB+9LG\nS9esXXP8uHfplAZsnKuAAtwLPFFVfzTy0IPAuSt5DgIPjLS/q10NdCPwQhsi+gxwc5Kr2snfm1ub\nBsqdvzRs4/QA9gO/DXw1yWOt7feADwH3J7kT+DZwe3vsIeAdwEnge8C7AarqbJIPAI+05d5fVWc3\nZS0kSRNbNwCq6m9Zffwe4KZVli/grjVe6whwZJICJUlbw08CS1KnDABJ6pQBIEmdMgAkqVMGgCR1\nygCQpE4ZAJLUKQNAkjo11r2AtDlG752zY9s2nt2/v+s6JM2WPYApGr13zizvozMvdUiaLXsAki5q\nUXuMi7pekzAAOuGtm7VRi9pjXNT1moRDQJ3odQOXtDZ7ACJLS0C/3WC90qL2GBd1vTbKANhiQ9rg\nhlLnohndRi4DftjapxnIQ9pOL0UP6zgJA2CLXWyD88h7Mov077XWDveHI9PT3Fm5Y1ys7WtcngOY\nA775JrMI/16LsA6Lqqf/G3sAkhZaL8NbG2EAzImhdD99M2ko2+o5bq9rcwhozsz7xjrv9Wl63BaG\nzx7AgvJIXdJ6DIAFtdGd/9C695I2ziEgrcreg7T47AFsAYdftFnskWkrGQBbwJ2/VnMpBwZuU9oK\nDgFJU+JOXPPGAJCkTjkEJHXKc1UyALQmT0AuNnf+a+tl2zcAFshWHdG5o1CvFn3bNwAWyKJvrNK4\nHN4ajyeBJS0cd/7jWTcAkhxJcjrJ10baXpfkWJIn2++rWnuSfCTJySRfSXL9yHMOtuWfTHJwa1ZH\nWyVLSz8aF9VsnPs/uOb48VmXogUxTg/gvwMHLmh7L/BwVe0FHm7zAG8H9rafQ8BHYSUwgLuBtwA3\nAHefCw1Jk5mno1tDadjWDYCq+hvg7AXNtwJH2/RR4LaR9o/Vis8DVybZCdwCHKuqs1X1PHCMV4aK\nGt9UGpp5CiWNb6PnAHZU1TMA7ffrW/su4OmR5ZZb21rtugjfVJK20mafBM4qbXWR9le+QHIoyYkk\nJ86cObOpxUmSfmyjl4E+l2RnVT3ThnhOt/Zl4NqR5XYDp1r7Wy9oX1rthavqMHAYYN++fauGxDzy\nsjNJQ7PRHsCDwLkreQ4CD4y0v6tdDXQj8EIbIvoMcHOSq9rJ35tb28Jw56/VXHP8uFdQaW6t2wNI\n8uesHL1fnWSZlat5PgTcn+RO4NvA7W3xh4B3ACeB7wHvBqiqs0k+ADzSlnt/VV14YllaOB4YDN9o\neC/arSHWDYCq+s01HrpplWULuGuN1zkCHJmoOmkNvdyrRfNl0QLdW0Fo0BbtDbnVPFelUQbAwPmG\n1iTcVjTKAJhz6w11+IaWtFEGwEC4o9coz4G8kr3hyXk3UGnA3OH9mP8Wk7MHMEAe6Wge2SsZHgNg\nQPwwkYbAg5PhcAhIkjplD0DaZA7RaSgMgEvgG12rcZvQUBgAl8A3uobAAxWtxXMA0oJz56+12AOQ\nBs7LL7VRBoC0IDzSn45FClwDQNKmW6Sd5FoWIXA9ByBpyyzCTnKR2QOQNFhe4XRpDABpAfWyY+xh\nHbeSAaDB62G8eVzeL0qT8ByAFoZHg9Jk7AFMqJeutSbjdqEhsgcwId/kWo3bhYbIAJCkTjkEJEkb\nNPQLEAwASVtq6DvJcQx1CNAAGIMn+KRLt1nvId+Pm8dzAGNwY5Pmh+/HzWMPQAulh+EGabMYAFpI\n0zhKdChCo4Z48GEASBvkzn96hhS2Q6kTDABJU7TRo+Qh7VSHxJPAkqbOHfp8sAewhiF1OTU9bhda\nJFMPgCQHgP8CXA7cU1UfmnYN4/BNPnxbcVLO7WLzDPGk6biGsm5TDYAklwP/Ffi3wDLwSJIHq+rr\n06xDfXGnPd/W+v9ZhN7WvNc/7R7ADcDJqnoKIMl9wK3AXATAImxwWt2lHJG5XWy9c/8/lwE/nGkl\nfZl2AOwCnh6ZXwbeMuUazuObuy/PvfTSed+atVYguF3MxiLu/FcLt3kZGpp2AGSVtjpvgeQQcKjN\nvpjkm1te1StdDXxnBn93Mwy19pnU/Ryrb5QT8t98ugZZd9v5Xw18Z5O2u4v5l+MsNO0AWAauHZnf\nDZwaXaCqDgOHp1nUhZKcqKp9s6xho4Za+1DrhuHWbt3TN2+1T/tzAI8Ae5Ncl+RVwB3Ag1OuQZLE\nlHsAVfVykn8PfIaVy0CPVNXj06xBkrRi6p8DqKqHgIem/XcnNNMhqEs01NqHWjcMt3brnr65qj1V\ntf5SkqSF472AJKlTBsAFkhxI8s0kJ5O8d9b1jCvJkSSnk3xt1rVMIsm1ST6X5Ikkjyd5z6xrGkeS\nVyf5YpK/a3X//qxrmkSSy5N8OcmnZ13LJJJ8K8lXkzyW5MSs6xlXkiuTfCLJN9q2/q9nXRM4BHSe\ndquK/8XIrSqA3xzCrSqS/ArwIvCxqvpXs65nXEl2Ajur6ktJfhp4FLht3v/NkwR4TVW9mGQb8LfA\ne6rq8zMubSxJ/iOwD3htVb1z1vWMK8m3gH1VNajPASQ5CvzPqrqnXQH5L6rqu7Ouyx7A+X50q4qq\n+mfg3K0q5l5V/Q1wdtZ1TKqqnqmqL7XpfwSeYOUT43OtVrzYZre1n0EcTSXZDfwacM+sa+lBktcC\nvwLcC1BV/zwPO38wAC602q0q5n5ntCiS7AHeDHxhtpWMpw2jPAacBo5V1SDqBv4Y+F2GeeeFAv46\nyaPtrgFD8HPAGeC/tWG3e5K8ZtZFgQFwoXVvVaGtkeSngE8Cv1NV/zDresZRVT+oqjex8on2G5LM\n/dBbkncCp6vq0VnXskH7q+p64O3AXW3oc95dAVwPfLSq3gz8P2Auzi8aAOdb91YV2nxtDP2TwMer\n6i9nXc+kWnd+CTgw41LGsR/49TaWfh/wtiT/Y7Ylja+qTrXfp4FPsTJsO++WgeWRHuInWAmEmTMA\nzuetKqasnUy9F3iiqv5o1vWMK8n2JFe26Z8EfhX4xmyrWl9Vva+qdlfVHla2789W1W/NuKyxJHlN\nu1CANoRyMzD3V71V1bPA00l+oTXdxJzcAt+vhBwx5FtVJPlz4K3A1UmWgbur6t7ZVjWW/cBvA19t\n4+kAv9c+MT7PdgJH25VjlwH3V9WgLqkcoB3Ap1aOGbgC+LOq+qvZljS2/wB8vB1YPgW8e8b1AF4G\nKkndcghIkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1Kn/D3n6iFcy8WrwAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadfed20b10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = GaussianKernelDensity( [0.5, 1.2, 4.2, 1.9, 5.4, 3.4], weights=[1, 1, 1, 3, 1, 1], bandwidth=0.2 )\n",
    "d.plot( n=100000, edgecolor='c', color='c', bins=100 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the Gaussian kernel density, there is a Triangle kernel density, which has a slightly different shape. Mostly I liked how they looked like pointy hats, so I implemented it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFDFJREFUeJzt3X+MZWV9x/H3113wB1ZBGVm6S7oY\nN7Zo0kImiN3EGLGAP+LyhyaYVjeGZP+hFtsmVvyH1B9JTRqxJpWEsJjVUpGgBmKJlgCT1klEZoGq\nsFI2aGHLrzELKJrqrnz7xzyDs+vdnTtzf5wfz/uVbPbec8+98zxznnM+53nuc85EZiJJqs+Lmi6A\nJKkZBoAkVcoAkKRKGQCSVCkDQJIqZQBIUqUMAEmqlAEgSZUyACSpUhubLsDxnHrqqbl169amiyFJ\nnbJ3796fZubMauu1OgC2bt3KwsJC08WQpE6JiP8ZZj2HgCSpUgaAJFXKAJCkShkAklQpA0CSKmUA\nSFKlDABJqpQBIEmVMgAkqVIGgCZq0/w8m+bnmy6GpAFafSsIdd+Thw41XQRJx2APQJIqZQBIUqUM\nAEmqlAEgSZUyACSpUgaAJsbpn1K7GQCaGKeASu1mAEhSpQwASaqUASBJlTIANBV+ISy1jwGgqfAL\nYal9DABJqtRQARARfx0R90fEDyPiKxHxkog4MyLuioiHIuKrEXFiWffF5fn+8vrWFZ9zRVn+YERc\nOJkqSZKGsWoARMRm4K+A2cx8I7ABuAT4DHBVZm4DngYuLW+5FHg6M18HXFXWIyLOKu97A3AR8IWI\n2DDe6kiShjXsENBG4KURsRF4GfA48DbgpvL6HuDi8nhHeU55/fyIiLL8hsz8VWb+GNgPnDt6FdRG\nfukrtd+qAZCZ/wv8I/AISwf+Z4G9wDOZebisdgDYXB5vBh4t7z1c1n/1yuUD3qOeGfSlb8zNGQxS\niwwzBHQKS2fvZwK/D5wEvGPAqrn8lmO8dqzlR/+8XRGxEBELi4uLqxVPHeNsIKk9hhkCejvw48xc\nzMxDwNeBPwVOLkNCAFuAx8rjA8AZAOX1VwIHVy4f8J4XZOY1mTmbmbMzMzPrqJIkaRjDBMAjwHkR\n8bIyln8+8ABwJ/Dess5O4Oby+JbynPL6HZmZZfklZZbQmcA24HvjqYYkaa1W/aPwmXlXRNwE3AMc\nBu4FrgH+DbghIj5Vlu0ub9kNfDki9rN05n9J+Zz7I+JGlsLjMHBZZv5mzPWRJA0plk7O22l2djYX\nFhaaLobWIebmjvlavvWtUyuHVKOI2JuZs6ut55XAklQpA0CSKmUASFKlDABNnReDSe2w6iwgaS02\nzc+verGXF4NJ7WAPQGPlwV3qDgNAkiplAEhSpQwASaqUASBJlTIAJKlSBoAkVcoAkKRKGQAaG6/w\nlbrFANDYeBGY1C0GgBphb0FqngGgRthbkJpnAEhSpQwASaqUASBJlTIAJKlSBoAkVcoAUGOcCio1\nywBQY5wKKjXLAJCkShkAklQpA0Bj4Xi+1D0GgMbC8XypewwASaqUASBJlTIAJKlSBoAkVcoA0Mic\nASR1kwGgkTkDSOomA0CSKmUAqFEOH0nNMQDUKIePpOYYAJJUqaECICJOjoibIuJHEbEvIt4cEa+K\niNsi4qHy/yll3YiIz0fE/oj4fkScs+Jzdpb1H4qInZOqlCRpdcP2AP4J+FZm/iHwx8A+4GPA7Zm5\nDbi9PAd4B7Ct/NsFXA0QEa8CrgTeBJwLXLkcGpKk6Vs1ACLiFcBbgN0AmfnrzHwG2AHsKavtAS4u\nj3cAX8ol3wVOjojTgQuB2zLzYGY+DdwGXDTW2kiShjZMD+C1wCLwxYi4NyKujYiTgNMy83GA8v9r\nyvqbgUdXvP9AWXas5ZKkBgwTABuBc4CrM/Ns4Bf8drhnkBiwLI+z/Mg3R+yKiIWIWFhcXByieJKk\n9RgmAA4ABzLzrvL8JpYC4ckytEP5/6kV65+x4v1bgMeOs/wImXlNZs5m5uzMzMxa6qKOirk5rweQ\nGrBqAGTmE8CjEfH6suh84AHgFmB5Js9O4Oby+Bbgg2U20HnAs2WI6NvABRFxSvny94KyTPJ6AKkB\nG4dc78PA9RFxIvAw8CGWwuPGiLgUeAR4X1n3VuCdwH7gl2VdMvNgRHwSuLus94nMPDiWWqgRm+bn\nPXBLHTZUAGTmfcDsgJfOH7BuApcd43OuA65bSwHVXh78pW7zSmBJqpQBIEmVMgAkqVIGgCRVygCQ\npEoZAJJUKQNAreHVwNJ0GQBqDa8rkKbLAJCkShkAWheHa6TuMwC0Lg7XSN1nAEhSpQwASaqUASBJ\nlTIAtGZ+ASz1gwGgNfMLYKkfDAC1ir0LaXoMALWKvQtpegwASaqUASBJlTIAtCaO0Uv9YQBoTRyj\nl/rDAJCkShkAklQpA0Ct4/cM0nQYAGodv2eQpsMAkKRKGQCSVCkDQENzbF7qFwNAQ3NsXuoXA0CS\nKmUASFKlDABJqpQBIEmVMgDUSjE356wjacI2Nl0Atd+m+flGZgA560iaLHsAWpUHYqmfDABJqtTQ\nARARGyLi3oj4Znl+ZkTcFREPRcRXI+LEsvzF5fn+8vrWFZ9xRVn+YERcOO7KSJKGt5YewOXAvhXP\nPwNclZnbgKeBS8vyS4GnM/N1wFVlPSLiLOAS4A3ARcAXImLDaMWXJK3XUAEQEVuAdwHXlucBvA24\nqayyB7i4PN5RnlNeP7+svwO4ITN/lZk/BvYD546jEpKktRu2B/A54KPA8+X5q4FnMvNweX4A2Fwe\nbwYeBSivP1vWf2H5gPeopZyKKfXXqgEQEe8GnsrMvSsXD1g1V3nteO9Z+fN2RcRCRCwsLi6uVjxN\nWNMzgAwgaXKG6QFsB94TET8BbmBp6OdzwMkRsXwdwRbgsfL4AHAGQHn9lcDBlcsHvOcFmXlNZs5m\n5uzMzMyaK6R+aTqApD5bNQAy84rM3JKZW1n6EveOzPxz4E7gvWW1ncDN5fEt5Tnl9TsyM8vyS8os\noTOBbcD3xlYTSdKajHIl8N8BN0TEp4B7gd1l+W7gyxGxn6Uz/0sAMvP+iLgReAA4DFyWmb8Z4edL\nkkawpgDIzDlgrjx+mAGzeDLz/4D3HeP9nwY+vdZCSpLGzyuBJalSBoCOyRk4Ur8ZADomZ+BI/WYA\nSFKlDABJqpQBIEmVMgAkqVIGgCRVygBQ6zkdVZoMA0ADtemg63RUaTIMAA3kQVfqPwNAkiplAEhS\npQwASaqUASBJlRrlD8KohzbNz/sFsFQJewA6ggd/qR4GgCRVygBQJ7TpwjSpLwyAHto0P7+uA2ab\nD7IOTUnj55fAPbTeg6UHWa3F8gnDE9u3N1yS6VqeKHHaCSd0vu4GwAhq3QE0eW1vWzXPFluudx/q\nbwCMoG0NoOadsk+6sB3bXr5JafMw6Xr4HcAYbJqfJ+bmGm8cte6UfbNyOzbdpnSkvu1jBsAY9KlL\nqHbpQptqw8mP1schoHWywWvcujD0A4PbfhfKPao+7vP2ANapbQ1+UOPsW4PtW31W6srBH9rX9qdl\nUL273iYNgJ4Y1DjXsqN2oSH3+cBzvLp1ZYilC2Vcr2PVrett0gBYo+UvfNtkHDte1xty33Vh+3Sh\njOt1vLp1OfgMgDVarZFvaOBsra+NU+2zWnuqsb11OfgMgDF7nnY1iCcPHerMEIKOrw3bcLW23aa2\nP01t2DbrYQB03LANr9Ydswvcht3X1W3jNNA1aGPKd7Xh6be6sA27NEtpnPpeb3sAa9DXhtDGYFO7\n9LXtr6ZvM+mOZgBMyDQaw1p/Rl+nsnVZFw4aXShjG3RxPzIAJmQajWGtP6OLDbTv1rpNmjgYd6GM\n47Zpfp4NLZvuPQl+ByA1ZD0Hyi6EeBfKuJo+1GEY9gCG0KezgT6cnfXFeg8ybkONy6oBEBFnRMSd\nEbEvIu6PiMvL8ldFxG0R8VD5/5SyPCLi8xGxPyK+HxHnrPisnWX9hyJi5+SqNV5PHjrE8+t436R2\n1FGuRu76mY3XNEx3G673d13rdupavYfpARwG/jYz/wg4D7gsIs4CPgbcnpnbgNvLc4B3ANvKv13A\n1bAUGMCVwJuAc4Erl0Ojrya1o47zc7vUWJd1PcS6ZJTfda3bqUv1XjUAMvPxzLynPP45sA/YDOwA\n9pTV9gAXl8c7gC/lku8CJ0fE6cCFwG2ZeTAznwZuAy4aa21aqO1nBF1qrJLGa03fAUTEVuBs4C7g\ntMx8HJZCAnhNWW0z8OiKtx0oy461vNXadqO1cZSnzYGk9qi1rdVU76EDICJeDnwN+Ehm/ux4qw5Y\nlsdZfvTP2RURCxGxsLi4OGzxJqZtZ8jjKM+Thw51poH20TjuKDuN7TeuttY1NdV7qACIiBNYOvhf\nn5lfL4ufLEM7lP+fKssPAGesePsW4LHjLD9CZl6TmbOZOTszM7OWuvTeOHf6rjTQPhrXAabtw4tq\nv2FmAQWwG9iXmZ9d8dItwPJMnp3AzSuWf7DMBjoPeLYMEX0buCAiTilf/l5QllWhbUNJ6ocutIku\nhVSXyjoOw1wIth34APCDiLivLPs48A/AjRFxKfAI8L7y2q3AO4H9wC+BDwFk5sGI+CRwd1nvE5l5\ncCy1mIBx3wRq1M+qrWFqeJvm53li+/axf+a4dCGklnWprOOwagBk5ncYPH4PcP6A9RO47BifdR1w\n3VoK2JRpz7VeLNcavAiYOeGE39mhJ1Getv1lsxpMIsgn0TbG/ZmTCKm260KdvRXEAJM6294wN/fC\nwX35oA8ccZHZ8h+UWVmG2s5K+qzWbVljvZf34zaHgAEwwKQa68q/FuZfVlIb9f3+98cyqXq3/Xfp\nvYAa4NDL6Lr4nUgXytz2A9akTLLebZ6tZQCok7p4oOpimceprQfBrlxTMQkGgDQFkz7ItPXgulJb\nD4JtLdc0GABH6cKOpO6Z9EFmHJ9v26+PAXCUms8G1G2jHMCn9eVv28bD21SWJhgA0gSN474/w+rK\nrZvbdJLVhb+tMEkGwApt3EDqtjYd7NSsNrYFA2CFNm4gSZoUA0CdZY/td3Xld9KVco5b2+ptABRt\n2zBaXdt7bE20qfX8TrpSznGrtd4reSuIom0bRt3X9jZV620fltVc92X2AKSeGXaqZdMHwCZ73fb4\nl9gDkCag6QPMoDvKLt9qHHjhTrRNajKAmvzZbbpDaPUBUHs3WJPRhjZ1dBmeH7BM09embVD9EFCb\nNoZUm6Z7SrWrPgCkcfOgNrxpn4BN88rs1crRBlUHQFs2gvrFXmU7tWm4ty3lqDoA2rIRJE1e2/b3\nNtwYr+oAUPc1vQNpdG04EDal6VCqNgBqbXB90/QOdDTb1fq0bTvWotoAsMFpEmxX6pJqA0Aap7bM\nLumySfae7JkNVmUA2Bg0bp75j26Sv0O3z2BVBoCNQWqnGk/OmqxzlQEgqZ0mcXLW9lBp8oTUAJBG\n1PYDTO260ONvqg1VdTO4Nl0JqP6wTWlUTbWhqnoA7qj9VPOFRH3ktpyeagLARtVvhnt/jGtbdm1q\nbhPHqGoCwAOEJsETi/bq2j7fRHmrCQBpErp2kJFWMgAktc6oPauu9symXe4qAqCrjUHtZruanFF7\nVl3tmU273FUEQFcbg9rNdqWuqyIAJHXPeqf3dr1nNs1pzb0PgK43BrWT7Wo61tPL6kPPbFp1mHoA\nRMRFEfFgROyPiI9N+uf1oTGofWxX6oOpBkBEbAD+GXgHcBbw/og4a5plUH95Vt5Pa9mufWoD06jL\ntO8FdC6wPzMfBoiIG4AdwAOT+GF9agxa3TTOyr2f1PSt5ffdp20zjbpMewhoM/DoiucHyrKJ6FNj\nUDvYptQn0+4BxIBlecQKEbuAXeXpcxHx4Ag/71TgpyO8v036VBeYUH0GNbApcNtM2IjbtXX1GdaA\neg9blz8Y5vOnHQAHgDNWPN8CPLZyhcy8BrhmHD8sIhYyc3Ycn9W0PtUF+lWfPtUFrE+bjbsu0x4C\nuhvYFhFnRsSJwCXALVMugySJKfcAMvNwRPwl8G1gA3BdZt4/zTJIkpZM/S+CZeatwK1T+nFjGUpq\niT7VBfpVnz7VBaxPm421LpGZq68lSeqd3t8KQpI0WC8DYNq3m5ikiLguIp6KiB82XZZRRcQZEXFn\nROyLiPsj4vKmyzSKiHhJRHwvIv6r1Ofvmy7TqCJiQ0TcGxHfbLoso4qIn0TEDyLivohYaLo8o4qI\nkyPipoj4UdmH3jzyZ/ZtCKjcbuK/gT9jadrp3cD7M3MiVxtPWkS8BXgO+FJmvrHp8owiIk4HTs/M\neyLi94C9wMUd3jYBnJSZz0XECcB3gMsz87sNF23dIuJvgFngFZn57qbLM4qI+Akwm5mdvAbgaBGx\nB/jPzLy2zKJ8WWY+M8pn9rEH8MLtJjLz18Dy7SY6KTP/AzjYdDnGITMfz8x7yuOfA/uY4JXgk5ZL\nnitPTyj/OntGFRFbgHcB1zZdFh0pIl4BvAXYDZCZvx714A/9DICp3m5C6xMRW4GzgbuaLcloypDJ\nfcBTwG2Z2eX6fA74KPB80wUZkwT+PSL2ljsMdNlrgUXgi2WI7tqIOGnUD+1jAKx6uwk1KyJeDnwN\n+Ehm/qzp8owiM3+TmX/C0lXt50ZEJ4fpIuLdwFOZubfpsozR9sw8h6W7D19WhlO7aiNwDnB1Zp4N\n/AIY+fvNPgbAqrebUHPKWPnXgOsz8+tNl2dcSnd8Drio4aKs13bgPWXc/AbgbRHxL80WaTSZ+Vj5\n/yngGywND3fVAeDAih7mTSwFwkj6GADebqKlypemu4F9mfnZpsszqoiYiYiTy+OXAm8HftRsqdYn\nM6/IzC2ZuZWlfeaOzPyLhou1bhFxUploQBkquQDo7Ey6zHwCeDQiXl8Wnc8YbqM/9SuBJ61vt5uI\niK8AbwVOjYgDwJWZubvZUq3bduADwA/KuDnAx8vV4V10OrCnzDx7EXBjZnZ++mRPnAZ8Y+mcg43A\nv2bmt5ot0sg+DFxfTmwfBj406gf2bhqoJGk4fRwCkiQNwQCQpEoZAJJUKQNAkiplAEhSpQwASaqU\nASBJlTIAJKlS/w/k2sUPAGP+wwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fadfee8c810>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "d = TriangleKernelDensity( [0.5, 1.2, 4.2, 1.9, 5.4, 3.4], weights=[1, 1, 1, 3, 1, 1], bandwidth=0.5 )\n",
    "d.plot( n=500000, edgecolor='c', color='c', bins=250 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Independent Components Distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, a univariate distribution isn't good enough. Many times data comes in the form of multiple, independent, dimensions, each with their own distribution. We can model this with a multivariate distribution which contains a tuple of independent components. \n",
    "\n",
    "A common example may be to look at data whose signal is normally distributed, and whose duration is modelled by an exponential. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "-3.612085713764219\n"
     ]
    }
   ],
   "source": [
    "d = IndependentComponentsDistribution([ NormalDistribution(10, 1), ExponentialDistribution(0.5) ])\n",
    "print d.log_probability( (11, 3) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also weight the distributions. In this example, lets say that the signal is far more important than the duration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "-6.449962780172767\n"
     ]
    }
   ],
   "source": [
    "d = IndependentComponentsDistribution([ NormalDistribution(10, 1), ExponentialDistribution(0.5) ], weights=[3, 1])\n",
    "print d.log_probability( (11, 3) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the point becomes more probable, because we care more about the good fit to the signal dimension (the normal distribution) than it poorly fits the duration (exponential) distribution. You must be careful with the weightings though, because they don't have to sum to 1. In this sense, `weights = [3,1]` is not the same as `weights = [0.75, 0.25]`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do updates in the same manner. Drawing sampels from a Gaussian with `mu = 12` and `sigma = 2`, and an exponential with `mu = 5` (0.2 in pomegranate notation, which is more prominent), we can recover the underlying distributions with 1000 points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[12.160898634047232, 2.0366665283768683] [0.20419702218012306]\n"
     ]
    }
   ],
   "source": [
    "data = numpy.zeros((1000, 2))\n",
    "data[:,0] = np.random.randn(1000) * 2 + 12 \n",
    "data[:,1] = np.random.exponential(5, 1000)\n",
    "d.fit(data)\n",
    "\n",
    "print d.parameters[0][0].parameters, d.parameters[0][1].parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multivariate Gaussian Distribution"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's not always good enough to have independent distributions, as many signals are heavily correlated. A simple multivariate Gaussian has been written, as this is the most common use case, with full covariance updates. You provide a vector of mus, and a covariance matrix, and you're good to go! Unfortunately, plotting is not natively supported for this distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.73749922  5.11476927 11.18206336 16.53134358 20.93125287]\n"
     ]
    }
   ],
   "source": [
    "d = MultivariateGaussianDistribution( np.arange(5) * 5, np.eye(5) )\n",
    "print d.sample()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.02377337215938058, 7.969241783095424, 16.00711337421143, 23.988157982680665, 32.022657302705106], [[0.9431449057331389, -0.001642776696889797, 0.017314356941345806, 0.01096538884848701, -0.03258674233349018], [-0.001642776696889797, 0.9767716288924349, 0.018965950499477913, -0.007272981779940892, 0.04475422098013223], [0.017314356941345806, 0.018965950499477913, 0.9370010959472275, 0.018596776777878403, -0.028357435495068785], [0.01096538884848701, -0.007272981779940892, 0.018596776777878403, 1.0080029155391967, 0.020456741158268413], [-0.03258674233348995, 0.04475422098004492, -0.028357435495010578, 0.020456741158268413, 1.0468826718300115]]]\n"
     ]
    }
   ],
   "source": [
    "data = np.random.randn(1000, 5) + np.arange(5) * 8\n",
    "\n",
    "d.fit(data)\n",
    "print d.parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.34675139  7.52499761 14.4882111  25.46624009 32.03538407]\n"
     ]
    }
   ],
   "source": [
    "print d.sample()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conditional Probability Table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tables are a different type of multivariate distribution. In essence, a multivariate discrete distribution. To use them, we define the values, and give probabilities which sum to 1 for each instance of the parents. They must be linked to parent distributions (which can also be conditional probability tables). Large networks of tables are called Bayesian networks!\n",
    "\n",
    "Lets code up the Monty Hall problem very quickly, without inference (see Bayesian Network introduction for the inference component)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A\tA\tA\t0.0\n",
      "A\tA\tB\t0.5\n",
      "A\tA\tC\t0.5\n",
      "A\tB\tA\t0.5\n",
      "A\tB\tB\t0.0\n",
      "A\tB\tC\t0.5\n",
      "A\tC\tA\t0.5\n",
      "A\tC\tB\t0.5\n",
      "A\tC\tC\t0.0\n",
      "B\tA\tA\t0.0\n",
      "B\tA\tB\t0.5\n",
      "B\tA\tC\t0.5\n",
      "B\tB\tA\t0.5\n",
      "B\tB\tB\t0.0\n",
      "B\tB\tC\t0.5\n",
      "B\tC\tA\t0.5\n",
      "B\tC\tB\t0.5\n",
      "B\tC\tC\t0.0\n",
      "C\tA\tA\t0.0\n",
      "C\tA\tB\t0.5\n",
      "C\tA\tC\t0.5\n",
      "C\tB\tA\t0.5\n",
      "C\tB\tB\t0.0\n",
      "C\tB\tC\t0.5\n",
      "C\tC\tA\t0.5\n",
      "C\tC\tB\t0.5\n",
      "C\tC\tC\t0.0\n"
     ]
    }
   ],
   "source": [
    "guest = DiscreteDistribution({'A': 0.33, 'B': 0.33, 'C': 0.33})\n",
    "prize = DiscreteDistribution({'A': 0.33, 'B': 0.33, 'C': 0.33})\n",
    "monty = ConditionalProbabilityTable(\n",
    "    [['A', 'A', 'A', 0.0],\n",
    "     ['A', 'A', 'B', 0.5],\n",
    "     ['A', 'A', 'C', 0.5],\n",
    "     ['A', 'B', 'A', 0.5],\n",
    "     ['A', 'B', 'B', 0.0],\n",
    "     ['A', 'B', 'C', 0.5],\n",
    "     ['A', 'C', 'A', 0.5],\n",
    "     ['A', 'C', 'B', 0.5],\n",
    "     ['A', 'C', 'C', 0.0],\n",
    "     ['B', 'A', 'A', 0.0],\n",
    "     ['B', 'A', 'B', 0.5],\n",
    "     ['B', 'A', 'C', 0.5],\n",
    "     ['B', 'B', 'A', 0.5],\n",
    "     ['B', 'B', 'B', 0.0],\n",
    "     ['B', 'B', 'C', 0.5],\n",
    "     ['B', 'C', 'A', 0.5],\n",
    "     ['B', 'C', 'B', 0.5],\n",
    "     ['B', 'C', 'C', 0.0],\n",
    "     ['C', 'A', 'A', 0.0],\n",
    "     ['C', 'A', 'B', 0.5],\n",
    "     ['C', 'A', 'C', 0.5],\n",
    "     ['C', 'B', 'A', 0.5],\n",
    "     ['C', 'B', 'B', 0.0],\n",
    "     ['C', 'B', 'C', 0.5],\n",
    "     ['C', 'C', 'A', 0.5],\n",
    "     ['C', 'C', 'B', 0.5],\n",
    "     ['C', 'C', 'C', 0.0]], parents=[guest, prize]\n",
    "    )\n",
    "\n",
    "print monty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.69314718056\n",
      "-inf\n"
     ]
    }
   ],
   "source": [
    "print monty.log_probability( ('A', 'A', 'C') )\n",
    "print monty.log_probability( ('A', 'B', 'B') )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reason which we need the parent distributions to be explicit distributions is so that we can have seamless conversion from the conditional table to the joint table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A\tA\tA\t0.0\n",
      "A\tA\tB\t0.0555555555556\n",
      "A\tA\tC\t0.0555555555556\n",
      "A\tB\tA\t0.0555555555556\n",
      "A\tB\tB\t0.0\n",
      "A\tB\tC\t0.0555555555556\n",
      "A\tC\tA\t0.0555555555556\n",
      "A\tC\tB\t0.0555555555556\n",
      "A\tC\tC\t0.0\n",
      "B\tA\tA\t0.0\n",
      "B\tA\tB\t0.0555555555556\n",
      "B\tA\tC\t0.0555555555556\n",
      "B\tB\tA\t0.0555555555556\n",
      "B\tB\tB\t0.0\n",
      "B\tB\tC\t0.0555555555556\n",
      "B\tC\tA\t0.0555555555556\n",
      "B\tC\tB\t0.0555555555556\n",
      "B\tC\tC\t0.0\n",
      "C\tA\tA\t0.0\n",
      "C\tA\tB\t0.0555555555556\n",
      "C\tA\tC\t0.0555555555556\n",
      "C\tB\tA\t0.0555555555556\n",
      "C\tB\tB\t0.0\n",
      "C\tB\tC\t0.0555555555556\n",
      "C\tC\tA\t0.0555555555556\n",
      "C\tC\tB\t0.0555555555556\n",
      "C\tC\tC\t0.0\n"
     ]
    }
   ],
   "source": [
    "print monty.joint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 18 non-zero entries in the table, each of equal weight. 1 / 18 is 0.556, so this makes sense. The joint distribution is it's own object, which can be stored and used later if desired."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'pomegranate.distributions.JointProbabilityTable.JointProbabilityTable'>\n"
     ]
    }
   ],
   "source": [
    "a = monty.joint()\n",
    "print type(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-inf"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.log_probability( ('C', 'A', 'A') )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both conditional and joint distributions can be trained in the same manner as the other distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A\tA\tA\t0.0\n",
      "A\tA\tB\t0.0\n",
      "A\tA\tC\t0.0\n",
      "A\tB\tA\tinf\n",
      "A\tB\tB\t0.0\n",
      "A\tB\tC\tinf\n",
      "A\tC\tA\t0.0\n",
      "A\tC\tB\t0.0\n",
      "A\tC\tC\t0.0\n",
      "B\tA\tA\t0.0\n",
      "B\tA\tB\t0.0\n",
      "B\tA\tC\t0.0\n",
      "B\tB\tA\tinf\n",
      "B\tB\tB\t0.0\n",
      "B\tB\tC\t0.0\n",
      "B\tC\tA\t0.0\n",
      "B\tC\tB\t0.0\n",
      "B\tC\tC\t0.0\n",
      "C\tA\tA\t0.0\n",
      "C\tA\tB\t0.0\n",
      "C\tA\tC\t0.0\n",
      "C\tB\tA\t0.0\n",
      "C\tB\tB\t0.0\n",
      "C\tB\tC\t0.0\n",
      "C\tC\tA\tinf\n",
      "C\tC\tB\tinf\n",
      "C\tC\tC\t0.0\n"
     ]
    }
   ],
   "source": [
    "data = [['A', 'B', 'C'],\n",
    "        ['B', 'B', 'A'],\n",
    "        ['A', 'B', 'A'],\n",
    "        ['C', 'C', 'B'],\n",
    "        ['C', 'C', 'A'],\n",
    "        ['C', 'C', 'A']]\n",
    "\n",
    "a.fit(data)\n",
    "print a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A\tA\tA\t0.333333333333\n",
      "A\tA\tB\t0.333333333333\n",
      "A\tA\tC\t0.333333333333\n",
      "A\tB\tA\t0.5\n",
      "A\tB\tB\t0.0\n",
      "A\tB\tC\t0.5\n",
      "A\tC\tA\t0.333333333333\n",
      "A\tC\tB\t0.333333333333\n",
      "A\tC\tC\t0.333333333333\n",
      "B\tA\tA\t0.333333333333\n",
      "B\tA\tB\t0.333333333333\n",
      "B\tA\tC\t0.333333333333\n",
      "B\tB\tA\t1.0\n",
      "B\tB\tB\t0.0\n",
      "B\tB\tC\t0.0\n",
      "B\tC\tA\t0.333333333333\n",
      "B\tC\tB\t0.333333333333\n",
      "B\tC\tC\t0.333333333333\n",
      "C\tA\tA\t0.333333333333\n",
      "C\tA\tB\t0.333333333333\n",
      "C\tA\tC\t0.333333333333\n",
      "C\tB\tA\t0.333333333333\n",
      "C\tB\tB\t0.333333333333\n",
      "C\tB\tC\t0.333333333333\n",
      "C\tC\tA\t0.666666666667\n",
      "C\tC\tB\t0.333333333333\n",
      "C\tC\tC\t0.0\n"
     ]
    }
   ],
   "source": [
    "monty.fit(data)\n",
    "print monty"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cases of conditional probability tables, if parent tuples haven't been seen before, it assumes a uniform distribution over them, explaining the prevalence of 0.3333... entries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Serialization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Distributions (except tables right now) support serialization to JSONs using `to_json()` and `Distribution.from_json( json )`. We can see a few examples easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"frozen\" :false,\n",
      "    \"class\" :\"Distribution\",\n",
      "    \"parameters\" :[\n",
      "        5.0,\n",
      "        2.0\n",
      "    ],\n",
      "    \"name\" :\"NormalDistribution\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print NormalDistribution( 5, 2 ).to_json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"frozen\" :false,\n",
      "    \"dtype\" :\"str\",\n",
      "    \"class\" :\"Distribution\",\n",
      "    \"parameters\" :[\n",
      "        {\n",
      "            \"A\" :0.5,\n",
      "            \"C\" :0.25,\n",
      "            \"B\" :0.25\n",
      "        }\n",
      "    ],\n",
      "    \"name\" :\"DiscreteDistribution\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print DiscreteDistribution( {'A': 0.5, 'B': 0.25, 'C': 0.25} ).to_json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"frozen\" :false,\n",
      "    \"class\" :\"Distribution\",\n",
      "    \"parameters\" :[\n",
      "        5.0,\n",
      "        2.0\n",
      "    ],\n",
      "    \"name\" :\"GammaDistribution\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print Distribution.from_json( GammaDistribution( 5, 2 ).to_json() )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"frozen\" :false,\n",
      "    \"class\" :\"Distribution\",\n",
      "    \"parameters\" :[\n",
      "        [\n",
      "            5.0,\n",
      "            6.0,\n",
      "            7.0,\n",
      "            3.0,\n",
      "            5.0,\n",
      "            7.0,\n",
      "            3.0,\n",
      "            3.0,\n",
      "            5.0,\n",
      "            2.0\n",
      "        ],\n",
      "        1.0,\n",
      "        [\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1,\n",
      "            0.1\n",
      "        ]\n",
      "    ],\n",
      "    \"name\" :\"GaussianKernelDensity\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print Distribution.from_json( GaussianKernelDensity( [5,6,7,3,5,7,3,3,5,2] ).to_json() )"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
